Display system and display device

ABSTRACT

An example of a display system includes a sensor, a projector, and control means. The sensor senses user information for calculating a state regarding sleep of a user. The user information is, for example, biological information such as pulse. The projector projects and displays a predetermined image. The projector, for example, projects the image upward to project and display the image on the ceiling. The control means controls the projector in accordance with a state regarding sleep, which is calculated on the basis of the user information.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of International Application No. PCT/JP2014/078827, filedon Oct. 29, 2014, is incorporated herein by reference.

FIELD

The technique shown here relates to display systems and display devicesused when a user is in bed.

BACKGROUND AND SUMMARY

Conventionally, systems for monitoring sleep stages of users haveexisted. For example, such system gives a stimulus to a user inaccordance with his/her sleep stage to bring the user to another sleepstage.

In the system used when a user is in bed, it is desired to use displaymeans that is easy to view by the user in bed.

Therefore, the present application discloses a display system and adisplay device, each of which is capable of providing a display that iseasy to view by a user in bed.

(1)

One example of a display system disclosed in the present specificationincludes a sensor, a projector, and control means. The sensor sensesuser information for calculating a sleep state of a user. The projectorprojects and displays a predetermined image. The control means causesthe projector to project and display an image related to the sleep statecalculated on the basis of the user information.

(2)

Another example of the display system disclosed in the presentspecification includes a sensor, a projector, timing specifying means,and control means. The sensor senses user information for calculating asleep state of a user. The projector projects and displays apredetermined image. The timing specifying means specifies a timingregarding awakening of the user on the basis of the user information.The control means causes the projector to start projection and displayof the image in accordance with the specified timing.

(3)

Another example of the display system disclosed in the presentspecification includes a sensor, a projector, control means, andcommunication means. The sensor senses user information for calculatinga sleep state of a user in bed. The projector projects and displays apredetermined image when the user is in bed. The control means controlsthe projector. The communication means performs communication with aserver through a network.

(4)

The control means may cause the projector to display the image relatedto the sleep state calculated on the basis of the user information, at atiming in accordance with the sleep state of the user, which is a timingspecified on the basis of the user information.

(5)

The control means may cause the projector to display the image relatedto the sleep state at a timing before the user awakens or a timing whenthe user awakens, which is a timing specified on the basis of the userinformation.

(6)

The control means may control power supply to the projector inaccordance with the sleep state of the user.

(7)

The control means may start power supply to the projector in accordancewith a timing regarding awakening of the user, which is a timingspecified on the basis of the user information.

(8)

The control means may change an image for inducing sleep onset of theuser and/or an image for inducing awakening of the user, in accordancewith the sleep state of the user, and may cause the projector to projectand display the image.

(9)

The display system may further include judgment means for judging, whenthe user awakens, whether or not the awakening is an awakening inmid-course of sleep. In this case, the control means may cause theprojector to project and display different images in a case where theawakening of the user is judged to be an awakening in mid-course ofsleep and in a case where the awakening of the user is judged not to bean awakening in mid-course of sleep, respectively.

(10)

The display system may further include evaluation means for performingan evaluation of sleep of the user on the basis of the user information.In this case, the control means may cause the projector to project anddisplay a result of the evaluation at a timing in accordance with thesleep state of the user, which is a timing specified on the basis of theuser information.

(11)

The display system may further include a light source which irradiatesthe user with light. In this case, the control means may startirradiation of the light source at a timing before the user awakens,which is a timing specified on the basis of the user information.

(12)

The control means may successively acquire the user information from thesensor, and may control the projector in real time in accordance withthe acquired user information.

(13)

The communication means may receive, from the server, service data forproviding the user with a network service in accordance with anevaluation regarding health of the user.

(14)

The projector may project and display an image based on the servicedata.

(15)

The communication means may transmit, to the server, the userinformation sensed by the sensor and/or information calculated from theuser information.

(16)

The communication means may receive an image content from the server. Inthis case, the projector may project and display the image content.

(17)

The projector may project and display the image on a ceiling above theprojector itself.

(18)

The sensor may sense biological information of the user in a state ofnot being in contact with the user.

(19)

The sensor may sense the biological information from the user who isaway from the display device within a predetermined range.

(20)

The sensor may emit radio waves or sound waves toward a subject to besensed and receive reflected waves, and may output the biologicalinformation on the basis of a result of the reception.

(21)

The display system may further include a camera and correction means.The camera captures a projection spot of the projector. The correctionmeans performs, on the image projected and displayed by the projector, acorrection that takes into consideration unevenness of the projectionspot, on the basis of an image of the projection spot captured by thecamera.

(22)

The correction means may perform, on the image projected and displayedby the projector, a correction that takes into consideration color ofthe projection spot, on the basis of the image of the projection spot.

(23)

The display system may further include a loudspeaker provided in thesame casing as that for the sensor and the projector.

The present specification discloses a display device including some ofthe respective means of (1) to (23) described above. Further, thepresent specification discloses a non-transitory computer-readablestorage medium having stored therein an information processing programwhich causes a computer of the display system or the display devicedescribed above to function as some of the respective means of (1) to(23) described above. Furthermore, the present specification disclosesan information processing method (display method) executed in thedisplay system or the display device described above.

According to the display system and the display device described above,the state of a user of a hand-held terminal can be evaluated.

These and other objects, features, aspects and advantages will becomemore apparent from the following detailed description when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a non-limiting example of theconfiguration of an information processing system according to thepresent embodiment;

FIG. 2 shows a non-limiting example of the detailed configuration of aterminal system 2;

FIG. 3 shows a non-limiting example of the exterior view of the terminalsystem 2;

FIG. 4 is a timing chart showing a non-limiting example of the flow ofthe operation of the information processing system;

FIG. 5 shows a non-limiting example of how the terminal system 2 isarranged;

FIG. 6 shows a non-limiting example of the operation of the terminalsystem 2 during a sleeping period;

FIG. 7 shows a non-limiting example of an image projected by a projectorand representing an evaluation result;

FIG. 8 is a functional block diagram showing a non-limiting example ofthe functional configuration for calculating health information in aprocessing section 11 of a hand-held terminal 5;

FIG. 9 is a flowchart showing a non-limiting example of the flow ofprocesses executed by the terminal system during the sleeping period;

FIG. 10 shows one example of the relationship between the behavior ofthe user during the waking period and various types of informationdetermined by the information processing system;

FIG. 11 is a timing chart showing a non-limiting example of the flow ofthe operation of the information processing system during the wakingperiod;

FIG. 12 is a flowchart showing a non-limiting example of the flow of anactivity information calculation process;

FIG. 13 shows one example of activity information calculated in thepresent embodiment;

FIG. 14 is a flowchart showing a non-limiting example of the flow of anenvironmental information calculation process;

FIG. 15 shows one example of environmental information calculated in thepresent embodiment;

FIG. 16 is a flowchart showing a non-limiting example of the flow of anemotion information calculation process;

FIG. 17 is a flowchart showing another a non-limiting example the flowof the emotion information calculation process;

FIG. 18 is a flowchart showing a non-limiting example of the flow of apreference information calculation process;

FIG. 19 is a flowchart showing one example of the flow of an informationpresentation process;

FIG. 20 is a flowchart showing a non-limiting example of the flow ofprocesses executed on the server.

FIG. 21 shows one example of data organization of user data stored inthe server in the present embodiment;

FIG. 22 shows one example of a table used for calculating a secondfatigue level;

FIG. 23 shows one example of a table used for calculating anenvironmental index;

FIG. 24 shows one example of a table used for determining a networkservice in accordance with an evaluation result;

FIG. 25 shows a non-limiting example of information presented on thehand-held terminal 5;

FIG. 26 shows one example of an image displayed on a display 17 when aninput is made with respect to a detail button 55 shown in FIG. 25;

FIG. 27 shows one example of an image displayed on the display 17 when aprivilege is given to the user; and

FIG. 28 shows one example of an image displayed on a terminal of anotification-subject user.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS 1.Configuration of Information Processing System

In the following, with reference to the drawings, an informationprocessing system, an information processing server, a storage mediumhaving stored therein an information processing program, and aninformation processing method according to the present embodiment willbe described. First, The overall configuration of the informationprocessing system according to the present embodiment will be described.FIG. 1 is a block diagram showing one example of the configuration of aninformation processing system according to the present embodiment. Asshown in FIG. 1, an information processing system 1 includes a terminalsystem 2 and a server 3. The terminal system 2 and the server 3 cancommunicate with each other through a network 4 such as the Internet anda mobile communication network. It should be noted that although only asingle terminal system 2 is shown in FIG. 1, the information processingsystem 1 in the present embodiment includes multiple terminal systemseach provided to a user.

The terminal system 2 calculates various types of information (healthinformation, activity information, environmental information, emotioninformation, and preference information described later) related to theuser, and uploads the information to the server 3. In the presentembodiment, various types of information are calculated (generated) atan appropriate timing throughout a single day (i.e., while the user isasleep and awake). Although details will be described later, while theuser is asleep, the terminal system 2 calculates, for example, as healthinformation related to the health and/or the body of the user,information related to sleep and fatigue. In addition, while the user isawake, the terminal system 2 calculates, for example, the activityinformation related to the activity of the user, the environmentalinformation related to the environment surrounding the user, emotioninformation related to emotion of the user, and preference informationrelated to hobbies and preference (hobby, liking, interests, life style,etc.) of the user.

The server 3 performs an evaluation regarding the quality of life (QOL)of the user on the basis of the information uploaded from the terminalsystem 2. Although details will be described later, the terminal system2 in the present embodiment performs an evaluation of health (includingfatigue and sleep), environment, and emotion of the user, and furtherperforms an evaluation regarding QOL as an overall evaluation.Furthermore, the server 3 provides the user with a network service inaccordance with these evaluation results. In the present embodiment, theserver 3 presents the user with the evaluation results, provides theuser with advice information, recommendation information (e.g.,information introducing a commodity), or content (e.g., music) inaccordance with the evaluation results, or gives the user a privilege inaccordance with the evaluation results.

The above described various types of information used for evaluating QOLof the user or used for the network service in accordance with theevaluations are sometimes referred to as “QOL factor information”,herein. More specifically, in the present embodiment, the healthinformation, the activity information, the environmental information,the emotion information, and the preference information are calculatedas the QOL factor information.

In the following, one example of the configuration of the terminalsystem 2 will be described. FIG. 2 shows one example of the detailedconfiguration of the terminal system 2. FIG. 3 shows one example of theexterior view of the terminal system 2. As shown in FIGS. 1 to 3, theterminal system 2 includes a hand-held terminal 5 and a base device 6.The hand-held terminal 5 is carried by the user. The base device 6 isplaced at, for example, the home of the user.

In the present embodiment, the hand-held terminal 5 is a hand-held typeinformation processing apparatus, and the base device 6 is a cradle thatis connectable to the hand-held terminal 5. As shown in FIG. 3, thehand-held terminal 5 can connect to the base device 6 in adetachable/reattachable manner. Although details will be describedlater, communication between the hand-held terminal 5 and the basedevice 6 becomes possible when the hand-held terminal 5 and the basedevice 6 are connected. In addition, the base device 6 has a function ofperforming charging with regard to the hand-held terminal 5, and, whenthe hand-held terminal 5 and the base device 6 are connected, chargingof the hand-held terminal 5 by the base device 6 becomes possible.

In another embodiment, a configuration in which the hand-held terminal 5and the base device 6 are detachably/reattachably connected via a cablemay be used. In still another embodiment, the communication between thehand-held terminal 5 and the base device 6 may be performed throughwireless communication such as radio wave communication and infraredcommunication.

First, the configuration of the hand-held terminal 5 in the presentembodiment will be described. The hand-held terminal 5 is a hand-heldtype information processing apparatus, and, in the present embodiment,is a multifunctional device such as, for example, a mobile phone, asmart phone, or a tablet terminal. Thus, the hand-held terminal 5 hassome of the various types of functions (input function, output (display)function, information processing function, network communicationfunction, telephone call function, camera function, etc.) included in ageneral multifunctional device. The network communication function is acommunication function realized through the Internet and/or acommunication function realized through a mobile communication network.The hand-held terminal 5 may be attained by installing predeterminedfunctions on an off-the-shelf multifunctional device. In the presentembodiment, the hand-held terminal 5 is used for, in addition to be usedas the multifunctional device described above, calculating the QOLfactor information and transmitting the QOL factor information to theserver 3. Furthermore, the hand-held terminal 5 may be an informationprocessing apparatus that can be worn by the user such as, for example,a wrist watch-type or goggle-type terminal (i.e., wearable terminal).

As shown in FIG. 2, the hand-held terminal 5 includes a communicationsection 10. The communication section 10 connects to the network 4 toperform communication with the server 3. In the present embodiment, thecommunication section 10 is a communication module having the functionof connecting to a mobile communication network (mobile phonecommunication network) to perform communication. For example, thecommunication section 10 performs communication with a communicationmethod in compliance with telecommunications standards of 3G ortelecommunications standards of 4G (including LTE (Long TermEvolution)). It should be noted that the method with which the hand-heldterminal 5 communicates with the server 3 may be a method with which acommunication module with Wi-Fi authentication performs communicationthrough a wireless LAN. In addition, the hand-held terminal 5 may have afunction of communicating with the server 3 through the mobilecommunication network and a function of performing communication withthe server 3 through the wireless LAN.

The hand-held terminal 5 includes a processing section 11. Theprocessing section 11 executes various types of information processingto be executed by the hand-held terminal 5. The processing section 11 isconnected to each section of 10, and 12 to 19 of the hand-held terminal5. The processing section 11 has a CPU (Central Processing Unit) and amemory. In the hand-held terminal 5, the various types of informationprocessing described above are executed as a result of the CPU using thememory and executing an information processing program stored in thehand-held terminal 5. In the present embodiment, the processing section11 executes, as the information processing, a process for calculatingthe QOL factor information described above, and a process for presentingthe user with the information (e.g., information related to the networkservice) received from the server 3, etc. When the hand-held terminal 5operates as a multifunctional device, the processing section 11 executesinformation processing for achieving various functions.

The hand-held terminal 5 includes an input/output interface, andfunctions as an information processing apparatus (input/output terminal)for allowing the user to input and browse information. Specifically, thehand-held terminal 5 includes an operation input section 12, a display17, and a loudspeaker 18. The operation input section 12 is an inputdevice of any type for accepting an operation input by the user. In thepresent embodiment, the operation input section 12 includes buttons anda touch panel formed on the display 17. In another embodiment, thehand-held terminal 5 may include, as the operation input section 12, asensor (acceleration sensor, gyro sensor) for sensing an attitude of thehand-held terminal 5.

The display 17, which is one example of the output device, displaysvarious types of images generated on the hand-held terminal 5 inresponse to an input with respect to the operation input section 12, anddisplays various types of images (images related to the network service)based on data received from the server 3. The loudspeaker 18, which isone example of the output device, outputs various types of soundsgenerated by the hand-held terminal 5 in response to an input withrespect to the operation input section 12, and outputs various types ofsounds (music and audio related to the network service) based on thedata received from the server 3.

The hand-held terminal 5 includes a sensor for sensing (acquiring)information for calculating the QOL factor information. In the presentembodiment, the hand-held terminal 5 includes a position sensing section13, an environment sensor 14, a microphone 15, and a camera 16.

The position sensing section 13 senses the position of the hand-heldterminal 5. In the present embodiment, the position sensing section 13senses the position by using the GNSS (Global Navigation SatelliteSystem). The position sensing section 13 is, for example, a GPS (GlobalPositioning System) sensor (GPS module). It should be noted that theposition sensing method by the position sensing section 13 may be anymethod, and the position sensing section 13 may sense the position byusing, for example, a beacon. Furthermore, for example, the positionsensing section 13 may calculate information (e.g., informationindicating at which floor of the building one is located) indicating thealtitude of the user by calculating the change in altitude based on asensing result from an air pressure sensor.

The environment sensor 14 senses the environment surrounding thehand-held terminal 5. In the present embodiment, the environment sensor14 includes a temperature sensor and a humidity sensor. In anotherembodiment, an air pressure sensor, an illumination sensor, a noisesensor, an odor sensor, or the like may be included in the environmentsensor 14. More specifically, the environment sensor 14 may be one thatsenses at least one of temperature, humidity, illumination intensity,atmospheric pressure, sound, and odor. Furthermore, in anotherembodiment, the microphone 15 may be used as a sensor for sensing noisein the surrounding area.

The microphone 15 senses sound in the surrounding area of the hand-heldterminal 5. Although details will be described later, the microphone 15in the present embodiment is used for calculating the emotioninformation. The microphone 15 may be used for accepting an audio inputwith respect to the hand-held terminal 5.

The camera 16 is used for capturing an image of the user, andcalculating the emotion information by using the captured image (detailsare described later). In the present embodiment, the camera 16 isdisposed on the same side (inner side) where the display 17 is disposedon the hand-held terminal 5 (see FIG. 3). Thus, the camera 16 isdisposed at a position enabling capturing an image of the user who isoperating the hand-held terminal 5.

The hand-held terminal 5 includes a connector 19 for forming anelectrical connection with the base device 6. In the present embodiment,when the hand-held terminal 5 is mounted on the base device 6 (see FIG.3), the connector 19 makes contact with a connector 21 of the basedevice 6. With this, communication between the hand-held terminal 5 andthe base device 6 becomes possible.

It should be noted that the hand-held terminal 5 includes a battery thatis not diagrammatically represented, and each section of the hand-heldterminal 5 operates by the power supplied from the battery. Althoughdetails will be described later, in the present embodiment, the batteryof the hand-held terminal 5 can be charged by the base device 6.

Next, the configuration of the base device 6 in the present embodimentwill be described. In the present embodiment, the base device 6 isdisposed, for example, at the bedroom of the user (see FIG. 5), and isused for sensing biological information related to sleep of the userwhile the user is in bed. Here, the biological information isinformation sensed from the body of the user. In the present embodiment,respiration, pulse, and body movement are acquired as the biologicalinformation. In addition, the base device 6 is used for presenting theuser in bed with content (e.g., content encouraging sleep onset of theuser) and information (information of evaluation results related tosleep).

The base device 6 includes a support section for detachably/reattachablysupporting the hand-held terminal 5. Specifically, as shown in FIG. 3, arecessed portion in accordance with the shape of one portion of thehand-held terminal 5 is formed on a casing (support section) of the basedevice 6. When the hand-held terminal 5 is inserted in this recessedportion, the hand-held terminal 5 becomes mounted on the base device 6.

As shown in FIG. 2, the base device 6 includes the connector 21. Whenthe hand-held terminal 5 is inserted in the recessed portion, theconnector 19 of the hand-held terminal 5 and the connector 21 of thebase device 6 are connected. As a result, communication between thehand-held terminal 5 and the base device 6 becomes possible, andcharging of the hand-held terminal 5 by the base device 6 becomespossible.

The base device 6 includes a Doppler sensor 24 which is one example ofthe sensor for sensing the biological information. The Doppler sensor24, by discharging microwaves and receiving reflected waves of thedischarged microwaves, senses a moving object based on a differencebetween the frequency of the discharged microwaves and the frequency ofthe received microwaves. In the present embodiment, (an emission section24 a of) the Doppler sensor 24 emits radio waves in the forwarddirection of the base device 6 (see FIG. 3). In the present embodiment,the subject to be sensed by the Doppler sensor 24 is the user, and bodymovement of the user is sensed by the Doppler sensor 24. Althoughdetails will be described later, analysis such as frequency analysisperformed on the sensed biological information (output waveforms of theDoppler sensor 24) allows further calculation of biological informationother than body movement such as respiration and pulse.

The base device 6 includes a power acquisition section 23 for acquiringpower from an external power supply. In the present embodiment, the basedevice 6 is (may be detachably/reattachably) connected to a power plugand an AC adapter via a power cord that is not diagrammaticallyrepresented. When the power plug is connected to an electrical outletwhich is an external power supply, power is supplied to the poweracquisition section 23 of the base device 6. The base device 6 operatesby the power from the external power supply acquired by the poweracquisition section 23. In addition, the power acquisition section 23performs charging of the hand-held terminal 5 by transmitting thesupplied power to the hand-held terminal 5 through the connector 21. Inanother embodiment, the base device 6 may include a battery, and powercharged in the battery may be transmitted to the hand-held terminal 5.Furthermore, in the present embodiment, although charging is performedin a mode in which power is supplied through the connector, in anotherembodiment, power may be supplied through non-contact charging.

The base device 6 includes a projector 25 for projecting an image on ascreen or a wall surface (including the ceiling). The projector 25 maybe any display device that displays an image on a surface (may beuneven) away from the base device 6 by projecting the image on thesurface. In the present embodiment, as shown in FIG. 3, the projector 25is formed on the base device 6 such that a light projection section(lens) 25 a faces upward, i.e., such that the image is projected upward.More specifically, in the present embodiment, the projector 25 projectsthe image on the ceiling. In the present embodiment, for example, theprojector 25 displays an image encouraging awakening of the user orsleep onset (content for sleep onset described later, etc.), anddisplays an image showing an evaluation result of sleep when the userawakens in the morning.

In the present embodiment, the base device 6 corrects the image to beprojected on the ceiling by using, if necessary, a technology ofso-called projection mapping. More specifically, the base device 6corrects the image such that an image in accordance with the unevennessand/or the color of the projection plane (ceiling) of the projector 25is displayed. For that purpose, the base device 6 includes a camera 27for correcting the image. As shown in FIG. 3, the camera 27 is formed onthe base device 6 in a direction that includes an image capturing rangeof the location where the image is to be projected by the projector 25.Thus, the camera 27 is provided so as to face the same direction(upward) as the projector 25. The method for correcting the image willbe described later.

The base device 6 includes a loudspeaker 26. The loudspeaker 26 is usedfor, for example, outputting a sound encouraging awakening or sleeponset of the user (content for sleep onset etc., described later).

The base device 6 includes a control section 22 that controls eachsection 23 to 27 of the base device 6. The control section 22 isconnected to each of the sections 21 and 23 to 27 of the base device 6.The control section 22 executes various types of control processesexecuted by the base device 6. The control section 22 has a memory and aCPU (Central Processing Unit). In the base device 6, the various typesof control processes are executed when the CPU uses the memory andexecutes information processing programs stored in the base device 6.For example, the control section 22 controls charging operation of thehand-held terminal 5 by controlling the power acquisition section 23. Inaddition, the control section 22 causes the projector 25 and/or theloudspeaker 26 to reproduce information and content to be presented tothe user on the base device. Furthermore, the control section 22transmits information sensed by the Doppler sensor 24 to the hand-heldterminal 5.

It should be noted that the base device 6 may have another configurationin addition to or instead of the configuration shown in FIG. 2 indicate.For example, the base device 6 may include an environment sensor, adisplay, an indirectional loudspeaker, a light source (illumination), anodor generation device, and the like (see “[8. Modifications]” describedlater.

Next, the configuration of the server 3 will be described. The server 3is formed of one or more information processing apparatuses (serverdevice). Herein, a “server” refers to a single information processingapparatus (server device) and also to a whole server device group(server system) when the server is formed of multiple server devices.

In the present embodiment, although the server 3 will be described as anintegral configuration, the server 3 may have a configuration includingmultiple server devices divided in accordance with function and/or role.For example, the server 3 may have a configuration including a dataserver that accumulates the QOL factor information acquired from thehand-held terminal 5 and a service server that conducts an evaluation onthe basis of the QOL factor information to provide a network service.Furthermore, when the server 3 performs a service of providing acommodity or the like (“commodity or the like” means service is alsoincluded in addition to a commodity, and is described similarly in thefollowing) as a part of the network service, the server 3 may have aconfiguration including a shop server for billing and providing thecommodity or the like.

2. Operation Outline of Information Processing System

Next, a general outline of the operation of the information processingsystem 1 will be described. FIG. 4 is a timing chart showing one exampleof the flow of the operation of the information processing system. Thetiming chart of FIG. 4 shows one example of the flow of operation of theinformation processing system in a single day. As shown in FIG. 4, inthe present embodiment, the information processing system 1 is used indifferent modes during nighttime (period in which the user is asleep)and daytime (period in which the user is awake).

In a sleeping period (typically, night), the hand-held terminal 5 is setin a state of being mounted on the base device 6. At this moment, thebase device 6 senses biological information of the user by the Dopplersensor 24 and transmits the biological information to the hand-heldterminal 5 connected to itself (step S1). During the sleeping period,the biological information is repeatedly sensed by the base device 6 andrepeatedly transmitted to the hand-held terminal 5. Although notdiagrammatically represented in FIG. 4, the base device 6 executes aprocess of presenting information and/or content to the user during thesleeping period by using the projector 25 or the like.

The hand-held terminal 5 calculates information (sleep information)related to the sleep of the user on the basis of the biologicalinformation acquired from the base device 6 (step S2). Although detailswill be described later, the sleep information includes an indexregarding sleep (sleep index). The sleep index is a numerical valuerepresenting, for example, sleep hours, sleep latency, mid-sleep awakehours, and sleep efficiency, etc.

At an appropriate timing, the hand-held terminal 5 performs an operationin accordance with the sleep information, in other words, an operationin accordance with the sleep state of the user. Although details will bedescribed later, for example, the hand-held terminal 5 sets its ownoperation mode to OFF-mode in response to sleep onset of the user andsets its own operation mode to ON-mode in response to awakening of theuser. For example, the hand-held terminal 5 controls reproduction ofcontent on the base device 6 in accordance with the sleep state andcontrols the information to be presented to the user by the base device6 in accordance with the sleep state.

In addition, the hand-held terminal 5 calculates, as the QOL factorinformation, the health information based on the biological informationacquired from the base device 6 (step S3). In the present embodiment,the hand-held terminal 5 calculates, as the health information,information including the sleep information and fatigue information. Thefatigue information contains a fatigue index related to fatigue of theuser. Although details will be described later, the fatigue index iscalculated as a numerical value representing the fatigue level of theuser while taking into consideration the sleep index. In the presentembodiment, the health information is calculated in response toawakening of the user (i.e., in response to ending of the sleepingperiod).

When the health information is calculated, the hand-held terminal 5transmits the calculated health information to the server 3. The server3 stores (accumulates) the received health information distinctively foreach user (each hand-held terminal).

As described above, during the sleeping period, the terminal system 2acquires the biological information from the user in sleep, andcalculates the health information (index related to fatigue and sleep).The calculated health information is stored in the server 3.

On the other hand, during the waking period (typically, daytime), thehand-held terminal 5 is removed from the base device 6 and is carried bythe user. At this moment, the hand-held terminal 5 calculates, as theQOL factor information, the activity information, the environmentalinformation, the emotion information, and the preference information(step S5). In FIG. 4, the process (step S5) for calculating the QOLfactor information is shown only once. However, ordinarily, the processis executed multiple times during a single waking period (one day) at anappropriate timing in the waking period. Thus, in the presentembodiment, the QOL factor information is calculated in accordance withvarious behaviors of the user while awake. For example, in response towalk movement of the user, the activity information indicating activitycontent (movement through walking) is calculated, and the environmentalinformation indicating the environment through which the user is movingis calculated. In addition, for example, in response to the userparticipating a meeting at work, the emotion information indicating theemotion of the user in the meeting is calculated. Furthermore forexample, in response to the user stopping by a gymnasium on the way homefrom work, the preference information indicating the user's interest tophysical exercise is calculated.

The hand-held terminal 5 transmits the calculated QOL factor informationto the server (step S6). It should be noted that the hand-held terminal5 may transmit the QOL factor information to the server 3 every time theQOL factor information is calculated, or transmit multiple sets of theQOL factor information to the server 3 in response to arrival of apredetermined timing. The server 3 stores (accumulates) the received QOLfactor information distinctively for each user (each hand-heldterminal).

The server 3 performs an evaluation based on the QOL factor informationacquired from the hand-held terminal 5 (step S7). This evaluation isperformed on the basis of the health information acquired duringnighttime (sleeping period), and the activity information, theenvironmental information, and the emotion information which areacquired during daytime (waking period). In the present embodiment, theserver 3 calculates, as a factor index for determining a QOL index, ahealth index based on the health information and the activityinformation, an environmental index based on the environmentalinformation, and an emotion index based on the emotion information (seeFIG. 10 described later). Furthermore, the server 3 calculates the QOLindex on the basis of these three factor indices. Details of calculationmethods for each of the indices will be described later. In the mannerdescribed above, the server 3 performs an evaluation related to health,environment, and emotion of the user, and performs an evaluationregarding QOL of the user on the basis of these evaluation results.

Furthermore, the server 3 provides a network service in accordance withthe evaluation results. More specifically, the server 3 specifies, onthe basis of the evaluation results (step S8), service content to beprovided, and transmits, to the hand-held terminal 5, data (servicedata) related to the specified service (step S9).

The network service to be provided may be any content. In the presentembodiment, the server 3 provides, to the terminal system 2 (thehand-held terminal 5), advice information and/or recommendationinformation in accordance with the evaluation results. The adviceinformation is information including an advice for improving the varioustypes of indices (QOL index, etc.) indicating the evaluation results. Inaddition, the recommendation information is information for introducinga recommended commodity or the like to the user for improving thevarious types of indices indicating the evaluation results.

In addition, the server 3 provides, as the network service, content inaccordance with the evaluation results with respect to the terminalsystem 2 (the hand-held terminal 5). The content is, for example,content for improving the various types of indices indicating theevaluation results, and is more specifically music for improvinginsomnia, video for resolving stress, and the like.

Furthermore, the server 3 gives, as the network service, a privilege inaccordance with the evaluation results to the user of the terminalsystem 2 (the hand-held terminal 5). This privilege may be a privilegerelated to the network service or may be a privilege related to thehand-held terminal 5. For example, the privilege may be a privilegerelated to a charge the user has to pay for the network service, morespecifically, may be points that can be used when purchasing a commodityor the like introduced by the recommendation information. Furthermore,the privilege may be a privilege related to a charge the user has to payfor using the hand-held terminal 5, more specifically, may be aprivilege regarding discount of a usage charge for the hand-heldterminal 5 (e.g., telephone call charges).

In the present embodiment, an evaluation process (step S7) and a serviceprovision process (steps S8 and S9) by the server 3 are executed at apredetermined timing in a single day, specifically, are executed at atiming of reaching a predetermined clock time. This predetermined clocktime may be preset on the server 3 side or may be set to a clock timeinstructed in advance by the user of the hand-held terminal 5. Inanother embodiment, the predetermined timing may be specified on thebasis of activity of the user. For example, the predetermined timing maybe a timing when the user has returned home, or may be a timing when theuser has left his/her workplace. These timings can be specified on thebasis of, for example, the activity information.

As described above, in the present embodiment, the informationprocessing system 1 calculates the health information related to theuser in sleep (step S3), and calculates the activity information, theenvironmental information, the emotion information, and the preferenceinformation related to the user while awake (step S5). Then, theinformation processing system 1 evaluates QOL of the user on the basisof these QOL factor information (step S7). With this, since varioustypes of information related to the user are calculated throughout theday, QOL of the user reflecting the behavior and state of the userthrough the day can be evaluated.

3. Operation of Terminal System During Sleeping Period

(3-1: Operation Example)

Next, an operation example of the terminal system during the sleepingperiod will be described. FIG. 5 shows one example of how the terminalsystem 2 is arranged. As shown in FIG. 5, in the present embodiment, thebase device 6 is disposed in the bedroom of the user. The base device 6is disposed around (bedside, etc.) the user. In addition, as shown inFIG. 5, during the sleeping period, the hand-held terminal 5 is mountedon the base device 6. Thus, the user mounts the hand-held terminal 5 onthe base device 6 when going to bed. In response, operation by theterminal system 2 during the sleeping period is started (see FIG. 6).

In the present embodiment, the base device 6 has a function of chargingthe hand-held terminal 5. As a result, since the user can be motivatedto mount the hand-held terminal 5 on the base device 6, the possibilityof the user forgetting mounting of the hand-held terminal 5 on the basedevice 6 can be reduced.

FIG. 6 shows one example of the operation of the terminal system 2during the sleeping period. When the hand-held terminal 5 is mounted onthe base device 6, the base device 6 initiates charging of the hand-heldterminal 5. Although not diagrammatically represented, the base device 6ends a charging operation in response the battery of the hand-heldterminal 5 being charged to the capacity thereof. When the hand-heldterminal 5 is mounted on the base device 6, if the remaining batterylevel of the hand-held terminal 5 is equal to or higher than apredetermined level (e.g., half of the battery capacity), the basedevice 6 may suspend the charging. This is because, in the presentembodiment, there are cases where the user mounts the hand-held terminal5 on the base device 6 for a purpose other than the purpose of charging(e.g., a purpose of sensing the biological information). For example,the hand-held terminal 5 may notify the remaining battery level to thebase device 6 in response to being mounted on the base device 6, and thebase device 6 may assess the necessity of charging on the basis of thenotified remaining battery level. Alternatively, in response to beingmounted on the base device 6, the hand-held terminal 5 may assess thenecessity of charging on the basis of the remaining battery level, andnotify the necessity of charging to the base device 6.

In addition, when the hand-held terminal 5 is mounted on the base device6, the base device 6 initiates a sensing operation by the Doppler sensor24 (see FIG. 6). A sensing result by the Doppler sensor 24 istransmitted to the hand-held terminal 5. The hand-held terminal 5calculates the biological information (pulse, respiration, and bodymovement of the user) on the basis of the sensing result, and calculatesthe sleep index on the basis of the biological information (detailsdescribed later). The sensing operation by the Doppler sensor 24 and acalculation process of the sleep index based on the sensing result arerepeatedly executed during the sleeping period. In the presentembodiment, the sensing operation by the Doppler sensor 24 is repeatedlyexecuted until the user is no longer sensed. In addition, thecalculation process of the sleep index is repeatedly executed until theuser awakens.

Although details will be described later, the hand-held terminal 5calculates, in real time, the sleep index representing the state ofsleep of the user. Specifically, the hand-held terminal 5 can determine,in real time, at least whether the user is asleep or awake, anddetermine the depth of the sleep. It should be noted that “calculate(determine) in real time” as described above is not limited to a strictmeaning of instantaneously calculating (determining), but is a meaningthat also includes calculating (determining) with a delay of aboutseveral seconds.

As described above, in the present embodiment, since the terminal system2 uses an unworn type sensor (the Doppler sensor 24) that can sense thebiological information even without having the sensor worn by the user,the biological information can be sensed without obstructing the user(without disturbing sleep of the user).

In addition, when the hand-held terminal 5 is mounted on the base device6, the base device 6 initiates reproduction of content for sleep onsetby using the projector 25 and the loudspeaker 26 (see FIG. 6). Thecontent for sleep onset is content for encouraging sleep onset of theuser, in other words, content that has an effect of encouraging sleeponset of the user. For example, the content for sleep onset is an imageof a starlit sky, a sound of water flowing through a river, and thelike. In the present embodiment, the content for sleep onset is contentformed of an image (video) and/or sound. It should be noted that thebase device 6 may first display a menu image by using the projector 25,and enable the user to select the content to be reproduced from the menuimage.

The content (e.g., content for sleep onset) to be reproduced on theterminal system 2 (the base device 6) during the sleeping period may bedetermined on the basis of the sleep index and/or the biologicalinformation (pulse, respiration, etc.) calculated during the sleepingperiod. For example, the terminal system 2 may determine, as the contentto be reproduced, a music having a tempo matching the tempo ofrespiration or pulse, which is the biological information. For example,the terminal system 2 may specify the ideal rhythm during sleep from thesleep index, and determine, as the content to be reproduced, a musicthat guides the rhythm of respiration or pulse of the user to become aspecific rhythm (e.g., a music having a tempo matching the rhythm). Inaddition, reproduction of the content may be controlled on the basis ofthe biological information and/or the sleep index. For example, theterminal system 2 may alter the tempo of the music to be reproduced soas to be a tempo matching the tempo of pulse of respiration andreproduce the music, or may alter the tempo of the music to a tempomatching an ideal rhythm during sleep and reproduce the music. At thismoment, the tempo of the content (music) to be reproduced may be alteredin real time in accordance with the tempo of respiration or pulsesuccessively sensed (or in accordance with an ideal rhythm successivelycalculated on the basis of the successively calculated sleep index).

In the present embodiment, as shown in FIG. 5, the projector 25 projectsan image (content for sleep onset) on the ceiling. Thus, the user caneasily see the image in a sleeping position.

As described above, the terminal system 2 can determine the sleep stateof the user depending on the mounting of the hand-held terminal 5 withrespect to the base device 6. In the present embodiment, the terminalsystem 2 performs various operations in accordance with the sleep stateof the user. In the following, specific examples of these variousoperations will be described.

(Alter Mode of Hand-Held Terminal 5 to OFF)

In the present embodiment, at a sleep onset (transition to a sleepstate) of the user, the operation mode of the hand-held terminal 5 isaltered (see FIG. 6). More specifically, at a sleep onset of the user,the hand-held terminal 5 determines that the user has entered a sleepstate through a process of calculating the sleep index. In response, thehand-held terminal 5 alters the operation mode thereof from ON-mode toOFF-mode.

Here, the ON-mode is a mode in which the calculation processes for theactivity information, the environmental information, the emotioninformation, and the preference information described above areexecuted. Furthermore, the ON-mode can be referred to as a mode in whichthe hand-held terminal 5 operates as a multifunctional device. While theuser is awake, the mode of the hand-held terminal 5 is basically set tothe ON-mode (as long as the user does not perform an operation ofaltering the mode).

On the other hand, the OFF-mode is a mode in which the calculationprocesses for the activity information, the environmental information,the emotion information, and the preference information described aboveare not executed. As described above, in the OFF-mode, power consumptioncan be reduced by not executing unnecessary processes during thesleeping period. It should be noted that, in the OFF-mode, communicationfunction with respect to the server 3 may be maintained and not shutdown or may be shut down.

In addition, in the OFF-mode, some of the functions among the functionsas the multifunctional device are shut down. In the present embodiment,in the OFF-mode, the function of notifying about an incoming telephonecall and E-mail (or any of those) by sound is shut down. With this, thepossibility of disturbing sleep of the user by the sound of thehand-held terminal 5 can be reduced. For example, in the OFF-mode, thehand-held terminal 5 replays a message of a telephone answering machineto an opponent side without sounding a ring tone when there is anincoming phone call, and does not sound a ring tone when there is anincoming mail.

(Reproduction Control of Content for Sleep Onset)

In response to the hand-held terminal 5 determining that the user hasentered the sleep state, the terminal system 2 changes the reproductionmode of the content for sleep onset (see FIG. 6). Specifically, theterminal system 2 shuts down image display (projection) by the projector25. This is because image display by the projector is unnecessary duringsleep. At this moment, the terminal system 2 may shut down power supplyto the projector 25. In addition, the terminal system 2 reduces theoutput sound volume of the loudspeaker 26. This is because the purposeand effect of outputting the sound related to the content for sleeponset are small (or zero) during sleep. However, in the presentembodiment, the possibility of the user waking up due to a sudden haltof the sound is taken into consideration, and the output sound volume isgradually reduced. In another embodiment, the terminal system 2 may shutdown sound output by the loudspeaker 26.

Then, when the sleep of the user becomes deep (e.g., enter a state ofnon-REM sleep), the hand-held terminal 5 determines that the depth ofthe sleep of the user has reached or exceeded a predetermined standardfrom the process of calculating the sleep index. In response, theterminal system 2 shuts down reproduction of the content for sleep onset(see FIG. 6). As a result, reproduction of content having a small effectcan be halted, and power consumption can be limited.

(Information Presentation at Time of Mid-Sleep Awakening)

In the present embodiment, when the user awakens (mid-sleep awakening)in mid-course of sleep, the terminal system 2 presents the user with thecurrent time (see FIG. 6). More specifically, when the user awakens inmid-course of sleep, the hand-held terminal 5 determines that the userhas undergone mid-sleep awakening by the process of calculating thesleep index. In response, the terminal system 2 displays the currenttime (on the ceiling) by using the projector 25. With this, when theuser wakes up in the middle of the night, the user can immediately knowthe current time. In addition, in the present embodiment, since thecurrent time is displayed on the ceiling, the user can know the currenttime while in the sleeping position.

It should be noted that the judgment of being an awakening in mid-courseof sleep (mid-sleep awakening) or an awakening at the time of arising(sometimes referred to as “arising awakening” for distinguishing from“mid-sleep awakening”) can be performed by, for example, setting ajudgment standard clock time. More specifically, the terminal system 2can judge an awakening to be a mid-sleep awakening when the awakening ofthe user is determined to be before the judgment standard clock time andto be an arising awakening when the awakening of the user is determinedto be at or after the judgment standard clock time. Here, the judgmentstandard clock time can be set based on a clock time at which the useris predicted to wake up (predicted arising clock time). For example, theterminal system 2 may set, as the judgment standard clock time, a clocktime that is a predetermined time period (e.g., two hours) earlier thanthe predicted arising clock time. Furthermore, the predicted arisingclock time may be calculated on the basis of past arising clock time(time of arising awakening) of the user, or may be set by the user. Whenthe user has set the time of an alarm clock with respect to the terminalsystem 2 (the hand-held terminal 5), the time may be set as thepredicted arising clock time.

(Reproduction Control of Content for Awakening)

In the present embodiment, when sleep of the user becomes shallow (e.g.,enter an REM sleep state) as the predicted arising clock timeapproaches, the terminal system 2 initiates reproduction of content forawakening (see FIG. 6). Specifically, in the case described above, thehand-held terminal 5 determines that the user has entered a state priorto awakening through the process of calculating the sleep index. Thedetermination of the user entering the state prior to awakening can bemade when, for example, the sleep has become shallow and the time iswithin a predetermined time period (e.g., one hour) from the predictedarising clock time. In response to the user being determined as enteringthe state prior to awakening, the terminal system 2 initiatesreproduction of the content for awakening. The content for awakening iscontent for encouraging awaken of the user, and is, in other words,content having an effect of encouraging awaken of the user. Examples thecontent for awakening include an image of bright blue sky, chirpingsound of a bird, and the like. In the present embodiment, the contentfor awakening is content formed of an image (video) and/or sound. Whenthe content for awakening is reproduced, the user can be encouraged towake up comfortably at an appropriate timing.

(Presentation of Evaluation Result at Time of Awakening)

In the present embodiment, when the user awakens (arising awakening),evaluation of the sleep of the user is performed and the evaluationresults are presented to the user (see FIG. 6). Specifically, when thehand-held terminal 5 determines that arising awakening has occurred fromthe method described above in “(Information Presentation at time ofMid-Sleep Awakening)”; the terminal system 2 performs an evaluation ofthe sleep of the user on the basis of (multiple sets of) the sleep indexrelated to the recent sleeping period. It should be noted that “thesleep index related to the recent sleeping period” refers to a sleepindex calculated on the basis of the biological information sensedduring the recent sleeping period. The terminal system 2 displays animage representing the evaluation result on the ceiling by using theprojector 25.

FIG. 7 shows one example of the image projected by the projector andrepresenting the evaluation result. In the present embodiment, theterminal system 2 calculates a pleasant sleep index number as theevaluation result. More specifically, as shown in FIG. 7, the terminalsystem 2 displays a pleasant sleep index number 31 calculated on thebasis of the sleep information. For example, the pleasant sleep indexnumber ranges from 0 to 100 points and is calculated such that thenumerical value is higher when the evaluation result is better (when thequality of the sleep is better). More specifically, the pleasant sleepindex number may be calculated by assigning an index specific weight toa predetermined one or more types of sleep indices. By displaying theevaluation result as a numerical value (score), the quality of the sleepcan be presented to the user in an easily understandable manner.

In addition, the terminal system 2 displays a face image 32 representingthe evaluation result (see FIG. 7). The face image 32 is displayed suchthat the display style thereof (specifically facial expression) ischanged depending on the pleasant sleep index number. For example, whenthe pleasant sleep index number is at a moderate level, a face imagerepresenting an ordinary facial expression is displayed; when thepleasant sleep index number is relatively high (the evaluation resultbeing good), a face image representing a smiley facial expression isdisplayed (see FIG. 7); and, when the pleasant sleep index number isrelatively low (the evaluation result being bad), a face imagerepresenting a fatigued facial expression is displayed. By displayingsuch a face image 32 as the evaluation result, the user can recognizethe evaluation result intuitively.

It should be noted that the terminal system 2 may display, as theevaluation result, a raw numerical value of the sleep index such as, forexample, sleep hours and sleep latency. Furthermore, a numerical valuedoes not necessarily have to be always displayed as the evaluationresult, and, for example, solely the face image 32 may be displayed asthe evaluation result.

In addition, the terminal system 2 displays advice information 33representing an advice in accordance with the evaluation result (seeFIG. 7). The advice information 33 represents an advice for improvingthe quality of the sleep. The advice information 33 is generated on thebasis of the sleep information and/or the sleep index number. Forexample, an advice information with different content may be displayeddepending on the magnitude of the sleep index number (magnitude of thenumerical value). The content of the advice information may bedetermined on the basis of various types of sleep indices such as, forexample, sleep hours, sleep latency, and mid-sleep awake hours.Furthermore, in another embodiment, recommendation information may bedisplayed instead the advice information 33 (or in addition to theadvice information 33).

In another embodiment, in addition to information of the evaluationresult being projected and displayed on the projector 25, otherinformation different from the information projected and displayed bythe projector 25 may be displayed on the display 17 of the hand-heldterminal 5. For example, information calculated from the evaluationresult may be displayed on the display 17. “Information calculated fromthe evaluation result” may be, for example, statistical information(e.g., information showing the change of the sleep index number for thepast one week) of the sleep index number which is the evaluation result,or may be information (e.g., the advice information and recommendationinformation described above) related to the network service based on theevaluation result.

(Alteration of Mode of Hand-Held Terminal 5 to ON/Transmission of HealthInformation)

In the present embodiment, when the user awakens (arising awakening),the operation mode of the hand-held terminal 5 is altered to the ON-mode(see FIG. 6). More specifically, when the hand-held terminal 5determines that arising awakening has occurred from the method describedabove in “(Information Presentation at time of Mid-Sleep Awakening)”;the hand-held terminal 5 alters the operation mode thereof from theOFF-mode to the ON-mode. As a result, while the user is awake, since thehand-held terminal 5 is automatically maintained in the ON-mode, theuser can use the hand-held terminal 5 without being obstructed.

In addition, when the user awakens (arising awakening), the hand-heldterminal 5 calculates the health information on the basis of the sleepinformation (sleep index), and transmits the health information to theserver 3 (see FIG. 6). The health information calculated here includesthe sleep information and the fatigue information (fatigue index)calculated from the sleep information. The calculation method for thefatigue index will be described later.

As described above, in the present embodiment, the terminal system 2performs various operations in accordance with the sleep state of theuser during the sleeping period of the user. When the sleeping periodends (when the user awakens), the user moves (e.g., exits the bedroom)and leaves the sensing range of the Doppler sensor 24. Thus, when theuser is no longer sensed by the Doppler sensor 24, the terminal system 2ends the sensing operation of the Doppler sensor 24 (see FIG. 6). As aresult, the terminal system 2 ends the operation performed during sleepof the user.

In another embodiment, when the base device 6 and the hand-held terminal5 are wirelessly communicable, the terminal system 2 may alter theoperation mode of the hand-held terminal 5 to the ON-mode in response toa subject being no longer sensed by the Doppler sensor 24. Morespecifically, when the subject is no longer sensed by the Doppler sensor24, the base device 6 sends a notification to the hand-held terminal 5,and the hand-held terminal 5 alters the operation mode to the ON-mode inresponse to receiving this notification.

The user, at the time of arising, may conceivably remove the hand-heldterminal 5 from the base device 6 to be carried. Thus, in anotherembodiment, the terminal system 2 may alter the operation mode of thehand-held terminal 5 to the ON-mode in response to the hand-heldterminal 5 being removed from the base device 6. More specifically, thehand-held terminal 5 may sense the connection state with the base device6 in the OFF-mode, and, when the connection with the base device 6 is nolonger sensed, may alter the operation mode of itself to the ON-mode.

In another embodiment, the terminal system 2 may end the sensingoperation of the Doppler sensor 24 in response to awakening (arisingawakening) of the user. Furthermore, in another embodiment, the terminalsystem 2 may end the sensing operation of the Doppler sensor 24 inresponse to the hand-held terminal 5 being removed from the base device6.

Still further, in another embodiment, a measurement period of thebiological information may be determined with any method, i.e.,initiating or ending the sensing by the Doppler sensor 24 may beperformed in accordance with any condition, and the following method maybe used, for example.

For example, in another embodiment, the terminal system 2 mayintermittently perform the sensing by the sensor (the Doppler sensor24), and may determine the measurement period on the basis of thesensing result. Specifically, the terminal system 2 judges whether ornot the user is sensed (whether or not the user exists within a sensingrange) by performing the sensing by the Doppler sensor 24 at apredetermined time interval. When the user is not sensed, the terminalsystem 2 shuts down the sensing by the Doppler sensor 24. In this case,the measurement is not initiated. On the other hand, when the user issensed, the terminal system 2 initiates the measurement by continuingthe sensing by the Doppler sensor 24. When the measurement is initiated,the terminal system 2 continues the measurement while the user is sensedby the Doppler sensor 24. Thus, similarly to the embodiments describedabove, the terminal system 2 ends the measurement by the Doppler sensor24 in response to the user being no longer sensed. As a result, themeasurement period by the Doppler sensor 24 can be determined on thebasis of the result of sensing (performed intermittently) by the Dopplersensor 24 itself. With this, since the measurement period can bedetermined without using other devices such as a sensor, the deviceconfiguration can be simplified.

In another embodiment, the measurement period with the Doppler sensor 24may be determined on the basis of a sensing result from a sensor (e.g.,human sensor) different from the Doppler sensor 24. For example, theterminal system 2 may use an infrared sensor and/or a camera as thehuman sensor. Specifically, the terminal system 2 intermittently orcontinuously senses the user by using the human sensor. During theperiod in which the user is sensed by the human sensor, the terminalsystem 2 performs measurement by the Doppler sensor 24. Also with this,the measurement period can be automatically determined similarly to themethod of the present embodiment and the method for determining themeasurement period on the basis of the sensing result from the Dopplersensor 24. Thus, since the user does not have to perform an operationfor (initiating and ending) the measurement, usability of the terminalsystem 2 can be improved. Furthermore, since the measurement of thebiological information can be performed without too much time and effortand without placing any burden on the user; continuous acquisition ofthe biological information can be performed easily.

In another embodiment, the measurement period with the Doppler sensor 24may be predetermined. For example, the terminal system 2 may perform themeasurement within a predetermined time zone (e.g., a time zone from20:00 during the night to 10:00 in the next morning), or may constantlyperform the measurement (as long as a shut-down operation is performedby the user). Similarly to above, also with this, since the user doesnot have to perform an operation for initiating and ending themeasurement, usability of the terminal system 1 can be improved.

In addition, in the embodiment described above, the terminal system 2automatically (in other words, without a specific operation by the user)calculates the health information (sleep index and fatigue index) basedon the biological information, in addition to automatically performingthe measurement of the biological information. Thus, since the healthinformation is calculated in the terminal system 2 without having theuser perform a special operation, usability of the terminal system 2 canbe improved.

(3-2: Calculation Method of Health Information)

In the following, a process for calculating the health information(sleep index and fatigue index) on the basis of the sensing result fromthe Doppler sensor 24 will be described. FIG. 8 is a functional blockdiagram showing one example of the functional configuration forcalculating health information in the processing section 11 of thehand-held terminal 5. As shown in FIG. 8, the processing section 11includes a waveform analysis section 41, a sleep calculation section 42,an autonomic nerve calculation section 43, and a fatigue calculationsection 44.

The waveform analysis section 41 calculates respiration, pulse, and bodymovement as additional biological information on the basis of thebiological information (output waveform) sensed by the Doppler sensor24. Conventionally, obtaining waveforms representing respiration, pulse,and body movement has been known to be possible through separation ofthe output waveform of the Doppler sensor 24 in accordance with thefrequency. The waveform analysis section 41 separates the outputwaveform through frequency analysis and the like into a frequencybandwidth corresponding to respiration, a frequency bandwidthcorresponding to pulse, and a frequency bandwidth corresponding to bodymovement, and individually outputs the separated waveform data. As shownin FIG. 8, an output from the waveform analysis section 41 is inputtedto both the sleep calculation section 42 and the autonomic nervecalculation section 43.

The sleep calculation section 42 calculates various types of sleepindices on the basis of the biological information (respiration, pulse,and body movement). Conventionally, methods for calculating the sleepindices on the basis of respiration, pulse, and body movement are known.In the present embodiment, the sleep calculation section 42 calculatessleep indices indicating the following information.

Sleep latency (sleep onset latency)

Mid-sleep awake hours

Number of times of mid-sleep awakening

Sleep efficiency

Total sleep hours

Activity level during sleep

Sleep stage

REM sleep hours

Non-REM sleep hours

Sleep quality

In another embodiment, only some of the sleep indices described abovemay be calculated, or a type of sleep index different from the sleepindices described above may be calculated.

The autonomic nerve calculation section 43 calculates an index(autonomic nerve index) indicating the action level of the autonomicnerves (sympathetic nerves and parasympathetic nerves) on the basis ofthe biological information. Specifically, the waveform of pulse (RRinterval) included in the biological information is frequency-analyzedby using maximum entropy method and Fourier transformation to calculatea high frequency component (approximately 0.15 to 0.40 [Hz]) HF and alow frequency component (approximately 0.04 to 0.15 [Hz]) LF of thewaveform. The high frequency component HF is known to indicate theaction level of parasympathetic nerves and the low frequency componentLF is known to indicate the action level of sympathetic nerves. Inaddition, it is known that the fatigue level can be evaluated from aratio between the action level of parasympathetic nerves and the actionlevel of sympathetic nerves (e.g., see Japanese Laid-Open PatentPublication No. 2010-201113). Thus, the autonomic nerve calculationsection 43 calculates a ratio (LF/HF) between the high frequencycomponent HF and the low frequency component LF as the autonomic nerveindex. As shown in FIG. 8, an output from the autonomic nervecalculation section 43 is used as an input for the fatigue calculationsection 44.

The fatigue calculation section 44 calculates the fatigue index on thebasis of the sleep indices and the autonomic nerve index. In the presentembodiment, as the fatigue index, a fatigue level indicating the degree(level) of fatigue with a numerical value ranging from 0 to 100 iscalculated. Although any method can be used for calculating the fatigueindex, for example, the following methods are conceivable. It should benoted that, hereinafter, the fatigue level calculated here is referredto as a “first fatigue level” in order to distinguish that from afatigue level corresponding to a waking period described later.

A first method is a method of calculating the fatigue index from thesleep indices. Here, the sleep indices are thought to correlate with thefirst fatigue level. For example, the following are examples in whichthe first fatigue level is speculated to be high.

Long sleep latency.

Long mid-sleep awake hours.

Frequent mid-sleep awakening.

Low sleep efficiency.

Short total sleep hours.

Upset of balance between REM sleep hours and non-REM sleep hours (theratio between REM sleep hours and non-REM sleep hours being outside thenormal range).

Thus, the fatigue calculation section 44 calculates the first fatiguelevel such that the first fatigue level becomes high when the sleepindices fit the examples described above and such that the first fatiguelevel becomes low when the sleep indices does not fit the examplesdescribed above. For example, the fatigue calculation section 44 mayjudge whether or not any of the items described above are satisfied,calculate points in accordance with the number of satisfied items, andcalculate the first fatigue level in accordance with the total points.At this moment, the fatigue calculation section 44 may calculateweighted points for each of the items. Furthermore, a reference value(e.g., “six hours” for the total sleep hours) may be set for each of theitems, and the points may be calculated such that the points becomelarger as the value of a calculated sleep index deviates more from thereference value.

As described above, in the present embodiment, the sleep indices relatedto sleep of the user is calculated on the basis of the biologicalinformation, and the fatigue index is calculated on the basis of thesleep indices. Since a correlation is thought to exist between the sleepindices and the degree of fatigue as described above, the accuracy ofthe fatigue index can be improved by calculating the fatigue index onthe basis of the sleep indices.

A second method is a method of calculating the fatigue index on thebasis of the autonomic nerve index. As described above, it is known thatthe first fatigue level can be evaluated by using the balance of theaction levels of sympathetic nerves and parasympathetic nerves, i.e.,the autonomic nerve index. Thus, for example, the fatigue calculationsection 44 calculates the first fatigue level such that the firstfatigue level becomes higher as the value of the autonomic nerve indexdeviates more from the reference value.

In the present embodiment, the fatigue calculation section 44 calculatesthe first fatigue level by use the first and second methods.Specifically, the fatigue calculation section 44 calculates respectivefatigue levels from the two methods, and calculates the final firstfatigue level on the basis of the respective calculated fatigue levels.The fatigue calculation section 44 may, for example, use an average ofthe two fatigue levels as the final first fatigue level, or maycalculate the final first fatigue level by assigning a weight to one ofthe two fatigue levels.

In another embodiment, in addition to the two methods described above,the first fatigue level may be calculated by using the method describednext. That is, a method of calculating the first fatigue level on thebasis of sleep hours within a predetermined period of time (e.g., oneweek) may be used. Conventionally, in a Fatigue Risk Management System(FRMS), a technique of calculating the fatigue level on the basis ofsleep hours and work hours exists. In the technique, for example, bysetting the work hours as a constant for simplification, the firstfatigue level can be calculated on the basis of sleep hours (alone).

In another embodiment, any method can be used for calculating thefatigue index, and any type content of the fatigue index can be used. Inanother embodiment, as the fatigue index, a value indicating the fatiguelevel for each type of fatigue may be calculated. For example, inanother embodiment, the fatigue index may be three types of values,i.e., a value indicating the level of acute fatigue, a value indicatingthe level of accumulative fatigue, and a value indicating the level ofmental fatigue.

In another embodiment, the health information may be calculated by usingthe sensing results from the camera 16 and/or the microphone 15. Forexample, biological information such as pulse and/or body movement canbe calculated on the basis of an image of the user captured by thecamera 16. Thus, the processing section 11 may calculate the sleepindices (and the fatigue index) by using the biological informationobtained from the imaged captured by the camera 16 in addition to (orinstead of) the biological information obtained from the sensing resultsof the Doppler sensor 24. Furthermore, the processing section 11 maytake into consideration a snoring sound sensed by the microphone 15 tocalculate the sleep indices.

(3-3: Specific Example of Processes Executed by Terminal System 2)

Next, a specific example of processes executed in the terminal system 2during the sleeping period will be described. FIG. 9 is a flowchartshowing one example of the flow of processes executed by the terminalsystem during the sleeping period. The series of processes shown in FIG.9 is initiated in response to the hand-held terminal 5 being mounted onthe base device 6. In another embodiment, the series of processesdescribed above may be initiated in response to starting ofcommunication between the hand-held terminal 5 and the base device 6, ormay be initiated in response to the user performing a predeterminedinitiation operation with respect to the hand-held terminal 5 or thebase device 6. Each of the processes shown in FIG. 9 is executed byeither one of, or in cooperation between the hand-held terminal 5 andthe base device 6.

In the present application, the process at each step in the flowchartshown in the drawing is merely one example, and, as long as a similarresult is obtained, the processing sequence of each step may be switchedor another process may be executed in addition to (or instead of) theprocess as each of the steps. Herein, although the process at each ofthe steps in the flowchart is described as to be executed by a CPU ofeach device (the hand-held terminal 5, the base device 6, and the server3); processes of one portion of the steps in the flowchart may beexecuted by a dedicated circuit or a processor other than a CPU.

First, at step S11, the terminal system 2 executes a start-up process.The start-up process is a process executed in response to the start ofthe series of processes shown in FIG. 9 (in the present embodiment, inresponse to the hand-held terminal 5 being mounted on the base device6). In the present embodiment, as the start-up process, a charginginitiation process, a sensing initiation process, and a reproductioninitiation process are executed (see FIG. 6). In another embodiment, anyprocess may be executed as the start-up process. One or two of the threeprocesses described above may be executed, another process differentfrom the three processes may be executed, or the start-up process may beeliminated and not executed.

In the charging initiation process, the base device 6 initiates chargingof the hand-held terminal 5. Specifically, the control section 22 givesan instruction to the power acquisition section 23 to initiate charging.In response to this instruction, the power acquisition section 23supplies power, supplied from an external power supply, to the hand-heldterminal 5 via the connector 21. It should be noted that the base device6 here is in a state of being connected to the external power supply(i.e., a state in which a power plug is connected to an electricaloutlet). The base device 6 may confirm the remaining battery level ofthe hand-held terminal 5, and initiate the charging operation under acondition that the remaining battery level is equal to or lower than apredetermined level. The charging operation started at step S11 ends inresponse to the battery of the hand-held terminal 5 being charged to thecapacity thereof.

In addition, in the sensing initiation process, the base device 6initiates, in order to calculate the health information, sensing by thesensor (the Doppler sensor 24) that senses the biological information.More specifically, the control section 22 of the base device 6 gives aninstruction to the Doppler sensor 24 to initiate the sensing operation.In response to this instruction, the Doppler sensor 24 initiates thesensing operation.

Furthermore, in the reproduction initiation process, the base device 6initiates reproduction of the content for sleep onset. Morespecifically, the control section 22 reads out the content for sleeponset stored in advance, and reproduces the content for sleep onset byusing the projector 25 and the loudspeaker 26. It should be noted thatthe content (content at sleep onset or content for awakening) reproducedon the base device 6 may be stored in the base device 6, may be storedin the hand-held terminal 5, or may be acquired from an external device(e.g., the server 3) via the hand-held terminal 5.

After the process at step S11, the processes at steps S12 to S20described in the following are repeatedly executed during the sleepingperiod. In the present embodiment, the process loop for steps S12 to S20is executed at a rate of once every predetermined time period.

At step S12 the terminal system 2 acquires the sensing result(biological information) from the Doppler sensor 24. The Doppler sensor24, which has initiated the sensing operation by the sensing initiationprocess at step S11, outputs the sensing result (output waveform) to thecontrol section 22. The control section 22 transmits the sensing resultto the hand-held terminal 5 (step S1 shown in FIG. 4). With this, thesensing result from the Doppler sensor 24 is acquired by the hand-heldterminal 5. The control section 22 may transmit information of thesensing result of the Doppler sensor 24 directly to the hand-heldterminal 5, or may process the sensing result in some way (e.g., processof removing noise included in signals of the sensing result, process ofcalculating the sleep index, etc.) and transmit the processed result tothe hand-held terminal 5.

At step S13, the hand-held terminal 5 calculates the sleep information(various types of sleep indices) (step S2 shown in FIG. 4). Morespecifically, the processing section 11 calculates the various types ofsleep indices on the basis of the sensing result (biologicalinformation) acquired at step S12. Calculation of the sleep indices isperformed by the method described in “(3-2: Calculation Method of HealthInformation)” above. At step S13, the processing section 11 only has tocalculate information (sleep indices) used for determining the sleepstate of the user at steps S14 and S15 described later. At step S13, theprocessing section 11 does not have to calculate the fatigue information(fatigue index) and sleep indices (e.g., total sleep hours), which canonly be calculated when the sleeping period ends.

At step S14, the hand-held terminal 5 controls output of the base device6 in accordance with the sleep state of the user (step S2 shown in FIG.4). More specifically, the processing section 11 controls output of thebase device 6 (output from the projector 25 and/or the loudspeaker 26)on the basis of the sleep indices calculated at step S13. Specifically,the processing section 11 judges whether or not the sleep state of theuser has entered a predetermined state on the basis of the sleep indicescalculated at step S13. Examples of the predetermined state include astate of sleep onset of the user, a state of being in a deep sleep, amid-sleep awakening state, a state prior to awakening, and the like (seeFIG. 6). The processing section 11, when judged that the predeterminedstate is obtained, executes a reproduction control process in accordancewith the state. In the present embodiment, with the process at step S14,the operations for controlling the power supply of the projector 25 andcontrolling reproduction of the content for sleep onset, the operationfor displaying the clock time in response to mid-sleep awakening, andthe operation for controlling reproduction of the content for awakening(see FIG. 6) are executed as described above.

In another embodiment, when content and/or information are/is to bepresented to the user during the sleeping period, the terminal system 2may perform an output from the hand-held terminal 5 in addition to orinstead of the output from the base device 6. For example, the displayof the current time performed in response to mid-sleep awakening of theuser may be performed on the hand-held terminal 5. In addition, forexample, the output of sound (music) in accordance with reproduction ofthe content by the projector 25 may be performed on the hand-heldterminal 5. It should be noted that an image identical to or differentfrom the image projected and displayed by the projector 25 may bedisplayed on the hand-held terminal 5.

At step S15, the hand-held terminal 5 controls the hand-held terminal 5in accordance with the sleep state of the user (step S2 shown in FIG.4). More specifically, the processing section 11 controls the operationmode of the hand-held terminal 5 on the basis of the sleep indicescalculated at step S13. Specifically, the processing section 11 judges,similarly to step S14, whether or not the sleep state of the user hasentered a predetermined state. The processing section 11, when judgedthat the predetermined state is obtained, alters the operation mode ofthe hand-held terminal 5 to a mode in accordance with the state. In thepresent embodiment, the altering operation to the OFF-mode and thealtering operation to the ON-mode are executed by the process at stepS15.

The process at step S15 may be a process for alter the settings of thehand-held terminal 5 regardless of the operation mode of the hand-heldterminal 5. For example, the hand-held terminal 5 may alter the settingsof the output sound volume and/or the settings of the screen display ofthe hand-held terminal 5 in accordance with the sleep state of the user.For example, when the user is in the sleep state, the output soundvolume of the hand-held terminal 5 may be set to zero, or the screendisplay may be set OFF.

At step S16, the hand-held terminal 5 judges whether or not the user hasawakened (arising awakening). More specifically, the processing section11 judges whether or not the user has awakened on the basis of the sleepindices calculated at step S13 from the method described in“(Information Presentation at time of Mid-Sleep Awakening)” above. Whenthe user is judged to be awake, the series of processes at steps S17 toS19 is executed. On the other hand, when the user is judged not to beawake, the series of processes at steps S17 to S19 is skipped and theprocess at step S20 is executed.

At step S17, the hand-held terminal 5 calculates the health informationon the basis of the information (biological information) acquired duringthe sleeping period (step S3 shown in FIG. 4). The health information iscalculated from the method described in “(3-2: Calculation Method ofHealth Information)” above. In the present embodiment, at step S17, theprocessing section 11 calculates the autonomic nerve index on the basisof the biological information acquired at step S12. In addition, theprocessing section 11 calculates the sleep indices (e.g., total sleephours, etc.) that cannot be calculated during the sleeping period on thebasis of the biological information acquired at step S12. Furthermore,the processing section 11 calculates the fatigue index (first fatiguelevel) on the basis of the various types of sleep indices and theautonomic nerve index. With this, the health information including thesleep indices and fatigue index is calculated. In another embodiment,the health information may include only one of the sleep information(sleep indices) and the fatigue information (fatigue index).

At step S18, the terminal system 2 presents the user with an evaluationresult of the sleep. More specifically, the processing section 11generates an image (see FIG. 7) indicating the evaluation result basedon the calculated sleep indices. In the present embodiment, the pleasantsleep index number and the advice information are calculated on thebasis of the sleep indices, and an image containing the calculatedinformation is generated. The processing section 11 transmits thegenerated image to the base device 6. The control section 22 of the basedevice 6 projects and displays the received image (on the ceiling) byusing the projector 25. With this, the image indicating the evaluationresult is presented to the user. In another embodiment, generation ofthe image may be performed by the base device 6. More specifically, thehand-held terminal 5 may transmit the information of the calculatedsleep indices to the base device 6, and generate the image on the basisof the sleep indices received by the base device 6.

At step S19, the hand-held terminal 5 transmits the calculated healthinformation to the server 3 (step S4 shown in FIG. 4). Morespecifically, the processing section 11 transmits, by the communicationsection 10, the health information calculated at step S17 to the server3. With this, the health information for a single sleeping period istransmitted to the server 3, and is saved in the server 3. In the mannerdescribed above, in the present embodiment, the hand-held terminal 5automatically transmits, to the server 3, the information that is to betransmitted (even without an instruction from the user). Thus, theinformation is uploaded to the server 3 without an instruction by theuser.

At step S20, the base device 6 judges whether or not the sensingoperation by the Doppler sensor 24 is to be ended. Specifically, thecontrol section 22 judges whether or not the user is no longer sensed(whether the user has left the sensing range of the sensor) on the basisof the sensing result from the Doppler sensor 24. When the user is nolonger sensed, the control section 22 judges that the sensing operationis to be ended, and ends the series of processes shown in FIG. 9. On theother hand, when the user is sensed, the control section 22 judges notto end the sensing operation, and executes the process at step S12 onceagain. Subsequently, unless a judgment to end the sensing operation ismade at step S20, the series of processes at steps S12 to S20 isrepeatedly executed.

In another embodiment, at step S19, the terminal system 2 may transmitthe health information to the server 3 and then end the series ofprocesses shown in FIG. 9. In this case, the process at step S12 isexecuted once again when the judgment result is negative at step S16,and the processes at steps S12 to S16 are repeatedly executed until thejudgment result at step S16 changes to positive.

In the present embodiment, when the hand-held terminal 5 is removed fromthe base device 6 for some reason (e.g., when the user rolls over in bedand accidently hits the hand-held terminal 5) during the sleepingperiod, the base device 6 cannot transmit the sensing result from theDoppler sensor 24 to the hand-held terminal 5. In this case, the basedevice 6 stores, in a storage section (memory, etc.) thereof, the dataof the sensing result that could not be transmitted to the hand-heldterminal 5. Then, in response to the next time when the hand-heldterminal 5 is mounted on the base device 6, the base device 6 transmits,to the hand-held terminal 5, the data of the sensing result stored inthe storage section. The hand-held terminal 5 that has received thedata, calculates the sleep indices on the basis of the sensing result(step S13). It should be noted that the hand-held terminal 5 does notnecessarily have to execute a control process (steps S14, S15) based onthe sleep indices calculated at this moment. This is because the sleepindices calculated here are based on past sensing results.

When the user is judged to be awake on the basis of the calculated sleepindices (when the judgment result at step S16 is positive), thehand-held terminal 5 executes the processes at steps S17 to S19. Asdescribed here, even when the hand-held terminal 5 is removed from thebase device 6 while the user is asleep, the health information iscalculated and transmitted to the server 3 in response to the next timewhen the hand-held terminal 5 is mounted on the base device 6. Thus, forexample, when the user awakens and notices that the hand-held terminal 5is removed from the base device 6, the user may mount the hand-heldterminal 5 on the base device 6. With this, the terminal system 2 canpresent the evaluation result of the sleep to the user and transmit thehealth information to the server 3.

In another embodiment, in a case where the hand-held terminal 5 and thebase device 6 are wirelessly communicable, the processes of steps S12 toS19 can be executed continuously even if the hand-held terminal 5 isremoved from the base device 6.

At step S14, an image is projected and displayed on the ceiling by theprojector 25. When the image is to be projected and displayed by theprojector 25, the base device 6 may execute a correction process forcorrecting the image. The correction process is a process for correctingthe image such that a proper image is projected and displayed, by takinginto consideration the unevenness and color of the projection spot(ceiling) of the projector 25. The base device 6 performs, on the imageto be projected and displayed by the projector 25, a correction thattakes into consideration the color and unevenness of a projection spoton the basis of an image of the projection spot captured by the camera27. Specifically, as shown in FIG. 9, before executing a process, thebase device 6 projects and displays a predetermined test image on theceiling by the projector 25. At this moment, when there is unevenness onthe ceiling, the projected and displayed test image will be distortedwhen compared to the original test image. In addition, when the ceilinghas a color, the projected and displayed test image will have adifferent color from the original test image. The base device 6 capturesan image of the test image that has been projected and displayed on theceiling by the camera 27 and that is distorted or has been changed incolor when compared to the original test image. Then, the base device 6specifies a correction process for correcting the original test imagesuch that the test image projected and displayed on the ceiling will bedisplayed properly, on the basis of the image captured by the camera 27.More specifically, the base device 6 specifies the content of thecorrection process for correcting the original test image such that theprojected and displayed test image is displayed in the correct shape atthe uneven projection spot and in the correct color at the projectionspot. When the correction process is specified, the base device 6 storesthe content of the correction process.

At step S14, when the image is to be projected and displayed by theprojector 25, the base device 6 corrects the image by the storedcorrection process, and projects and displays a corrected image by theprojector 25. With this, the projected and displayed image is restoredto a proper shape (the same shape as the original image) with the propercolor (the same color as the original image). The correction processdescribe above may be performed by a method similar to the technique forconventional projection mapping. In addition the correction process maybe a correction that takes into consideration only one of the unevennessand color of the projection spot.

4. Operation of Terminal System During Daytime

(4-1: Operational General Outline)

Next, a general outline of an operation of the information processingsystem during the waking period will be described. FIG. 10 shows oneexample of the relationship between the behavior of the user during thewaking period and various types of information determined by theinformation processing system. FIG. 11 is a timing chart showing oneexample of the flow of the operation of the information processingsystem during the waking period. FIG. 10 shows, as an example of thebehavior of the user in a single day, a case in which the user goes tooffice, attends a meeting, stops by at the gymnasium after leavingoffice, and returns home. Described in the following as an example is anoperation of the information processing system when user behaves in themanner as shown in FIG. 10 as a single day behavior.

As described above, in the present embodiment, the operation mode of thehand-held terminal 5 is altered to the ON-mode in response to awakeningof the user. In response to the operation mode set to the ON-mode, thehand-held terminal 5 initiates a process for calculating the QOL factorinformation (activity information, environmental information, emotioninformation, and preference information) that should be calculatedduring the waking period. More specifically, the hand-held terminal 5initiates sensing of positional information by the position sensingsection 13, and sensing of environment sensor information by theenvironment sensor 14 (step S21 shown in FIG. 11). Although details willbe described later, the positional information is used for calculatingthe activity information, and the environment sensor information is usedfor calculating the environmental information. In the presentembodiment, sensing of the positional information and sensing of theenvironment sensor information are repeatedly executed in the ON-mode(see FIG. 10).

In the example shown in FIG. 10, the user leaves home after awakening,and goes to his/her workplace. At this moment, the hand-held terminal 5calculates, on the basis of the positional information, activityinformation indicating that the user has moved (step S22 shown in FIGS.10 and 11). More specifically, when the sensed position indicated by thepositional information starts moving from a certain location (here,home) to another location (here, workplace) and stops, the hand-heldterminal 5 calculates the activity information indicating the movement.

Although the content of the activity information indicating the movementmay be in any form, activity information including a movement method anda movement amount is calculated in the present embodiment. Althoughdetails will be described later, as the movement method, either one ofwalking, train, and automobile is specified. As the movement amount, amoved distance and/or moving hours are/is calculated. In anotherembodiment, the activity information indicating the movement may includeinformation indicating the location prior to movement (departurelocation; home in the example of step S22) and/or a location aftermovement (destination; workplace in the example of step S22).

In the present embodiment, when the activity information is calculated,the hand-held terminal 5 calculates the environmental informationcorresponding to the activity information. The “environmentalinformation corresponding to the activity information” is environmentalinformation indicating the environment of the user in a period duringwhich the activity indicated by the activity information has beenperformed. At step S22, environmental information indicating theenvironment of the user in a period during which the movement has beenperformed is calculated. Although details will be described later, inthe present embodiment, the hand-held terminal 5 calculatesenvironmental information (see FIG. 15) including information (activitycontent shown in FIG. 15) indicating the content of the activityperformed by the user and information (environmental values shown inFIG. 15) indicating average ambient temperature and humidity sensedduring the activity.

When the activity information and the environmental information arecalculated as described above, the hand-held terminal 5 transmits thecalculated activity information and environmental information to theserver 3 (step S23). The server 3 stores (accumulates) the receivedactivity information and environmental information distinctively foreach user (each hand-held terminal). Also in the subsequent processes atsteps S26, S28, S30, S32, and S34; the server 3, when the QOL factorinformation is received from the hand-held terminal 5, stores theinformation distinctively for each user, similarly to step S23.

As shown in FIG. 10, the user attends a meeting after arriving at theoffice. At this moment, the hand-held terminal 5 determines the emotionof the user during the meeting. More specifically, the hand-heldterminal 5 calculates the emotion information indicating the emotionduring the meeting, and transmits the calculated emotion information tothe server 3 (steps S24 to S26 shown in FIGS. 10 and 11). Thecalculation of the emotion information is performed as described next.

First, in response to the start of the meeting, the hand-held terminal 5initiates sensing of information for calculating the emotion information(step S24). In the present embodiment, the judgment of whether or notthe meeting has been started is performed on the basis of scheduleinformation stored in the hand-held terminal 5. More specifically,schedule information indicating the date and time when the meetingstarts and ends is preregistered in the hand-held terminal 5. Thehand-held terminal 5 judges whether or not the meeting has started byreferring to the schedule information, and initiates the sensing whenthe meeting starts.

At step S24, the hand-held terminal 5 senses, as information forcalculating the emotion information, the sound sensed by the microphone15. More specifically, the hand-held terminal 5 initiates sound sensingby the microphone 15.

When the meeting ends, the hand-held terminal 5 ends sensing of theinformation (step S25). More specifically, The hand-held terminal 5 endssensing of sound by the microphone 15. It should be noted that thejudgment of whether or not the meeting has ended can be performed on thebasis of the schedule information stored in the hand-held terminal 5.

When the meeting ends, the hand-held terminal 5 calculates the emotioninformation on the basis of the information sensed during the meeting(information of sound sensed by the microphone 15) (step S26). Althoughdetails will be described later, in the present embodiment, thehand-held terminal 5 determines the emotion of the user on the basis ofvoice and the like of the user sensed by the microphone 15, andcalculates, as the emotion information, information indicating thedetermination result. Although details will be described later, in thepresent embodiment, emotion information representing the level (emotionintensity) of five types of emotions of anger, joy, sadness, hate, andpleasure is calculated.

As described above, when the user attends the meeting, sound during themeeting is sensed, and the emotion of the user is determined on thebasis of sensed sound. Although not diagrammatically represented inFIGS. 10 and 11, in the present embodiment, other than during themeeting, the emotion of the user is determined also while the user isoperating the hand-held terminal 5. Although details will be describedlater, while the user is operating the hand-held terminal 5, thehand-held terminal 5 captures an image of the user by using the camera16, and determines the emotion of the user on the basis of facialexpression of the captured user.

In the example shown in FIG. 10, the user leaves the office after themeeting ends. At this moment, the hand-held terminal 5 calculatesactivity information indicating work (has been done) (step S27 shown inFIGS. 10 and 11). More specifically, when the sensed position indicatedby the positional information starts moving from a specific location(here, workplace), the hand-held terminal 5 determines that the user hasstayed at the specific location, and calculates the activity informationactivity (here, work) corresponding to the specific location. Althoughany content may be used as the content of the activity informationindicating the activity corresponding to the location, in the presentembodiment, activity information including activity content (work) andactive hours (work hours) is calculated (see FIG. 13).

When the activity information indicating work is calculated as describedabove, the hand-held terminal 5 calculates the environmental informationcorresponding to the activity information. The calculation method of theenvironmental information is similar to the process at step S22. Morespecifically, at step S27, the hand-held terminal 5 calculates theenvironmental information indicating the environment of the user in aperiod during which the user is working.

When the activity information and the environmental information arecalculated as described above, the hand-held terminal 5 transmits thecalculated activity information and environmental information to theserver 3 similarly to the process at step S23 (step S28).

In the example shown in FIG. 10, the user stops by at the gymnasiumafter leaving office. When the user arrives at the gymnasium, thehand-held terminal 5 calculates, on the basis of the positionalinformation, activity information indicating that the user has moved(step S29 shown in FIGS. 10 and 11). The process for calculating theactivity information indicating the movement is similar to the processat step S22. In addition, similarly to step S22, the hand-held terminal5 calculates the environmental information corresponding to the activityinformation, and transmits the calculated activity information andenvironmental information to the server 3 (step S30).

In the example shown in FIG. 10, the user leaves the gymnasium afterfinishing the activity (physical exercise, etc.) at the gymnasium. Atthis moment, the hand-held terminal 5 determines that the user hasstayed at a specific location (here, gymnasium) by a process similar tothat at step S27. Then, the hand-held terminal 5 calculates thepreference information on the basis of the location where the user hasstayed, and transmits the calculated preference information to theserver 3 (step S32).

The calculated preference information may be information in any formatindicating hobby or preference of the user. In the present embodiment,the preference information represents a genre determine to be ofinterest of the user, among a plurality of genres (items). Examples ofthe plurality of genres include physical exercise, health, cosmetics,fashion, music, movies, news, cooking, meal, pet, game, and the like.Although details will be described later, the hand-held terminal 5stores, in advance, a table showing the association between location(facility) and the genre described above. When the user is determined tohave stayed at a specific location (facility), the hand-held terminal 5specifies a genre associated with the location in the table. Forexample, in the table, the genres of “physical exercise” and “health”are associated with gymnasium. Thus, when the user stops by at thegymnasium, the preference information representing “physical exercise”and “health” is calculated (generated).

Although not diagrammatically represented in FIGS. 10 and 11, in thepresent embodiment, the preference information is in some casescalculated on the basis of information inputted by the user (detailswill be described later). For example, when the user browses a web pageon the Internet by using the hand-held terminal 5, the hand-heldterminal 5 may determine the preference on the basis of search phrasesinputted to a search engine website by the user, or the preference maybe determined on the basis of the content of the browsed web page (e.g.,keywords, etc., contained in the web page).

In the example shown in FIG. 10, the user returns home after leaving thegymnasium. When the user returns home, the hand-held terminal 5calculates, on the basis of the positional information, the activityinformation indicating that the user has moved (step S33 shown in FIGS.10 and 11). The process for calculating the activity informationindicating the movement is similar to the process at step S22. Inaddition, similarly to step S22, the hand-held terminal 5 calculates theenvironmental information corresponding to the activity information, andtransmits the calculated activity information and environmentalinformation to the server 3 (step S34).

In the present embodiment, when a predetermined evaluation timingarrives, the server 3 performs a QOL factor information-based evaluationof the user on the basis of the QOL factor information transmitted fromthe hand-held terminal 5 (step S36 shown in FIG. 11, step S7 shown inFIG. 4). More specifically, the server 3 calculates various types ofindices (health index, environmental index, emotion index, and QOLindex) on the basis of various types of QOL factor information (healthinformation, activity information, environmental information, andemotion information) received from the hand-held terminal 5. The QOLfactor information-based evaluation of the user is performed for eachuser (user receiving a network service) having a hand-held terminal(step S36).

As shown in FIG. 10, in the present embodiment, the health indexindicating the health (fatigue level) of the user is calculated on thebasis of the health information and the activity information, theenvironmental index indicating the environment surrounding the user iscalculated on the basis of the environmental information, and theemotion index indicating the emotion of the user is calculated on thebasis of the emotion information. In addition, a QOL index which is acomprehensive index is calculated on the basis of the health index, theenvironmental index, and the emotion index described above. Details ofthe calculation method for these indices will be described later.

Although the evaluation timing when the server 3 performs the evaluationmay be any timing, in the present embodiment, the evaluation timing is atiming when a predetermined clock time determined in advance hasarrived. The evaluation timing may be a timing set by the server 3(provider of network service), or may be a timing specified by the user.In addition, the evaluation timing is not limited to the timing when thepredetermined clock time has arrived, and may be set on the basis of theactivity (the activity information) of the user. For example, theevaluation timing may be a timing when the user has left office, or maybe a timing when the user has returned home.

In addition, the number of times the server 3 performs the evaluation isnot limited to once per day, and may be multiple times per day. Forexample, the server 3, every time the QOL factor information is receivedfrom the hand-held terminal 5, may perform the evaluation based on thereceived QOL factor information. At this moment, the server 3 mayexecute an evaluation other than the overall evaluation (calculation ofan index other than the QOL index) every time the QOL factor informationis received, and may execute the overall evaluation (calculation of theQOL index) once per day at a predetermined evaluation timing. Inaddition, the server 3 may, if QOL factor information satisfying apredetermined condition is received (e.g., when a predetermined type ofQOL factor information is received), perform the QOL factorinformation-based evaluation in accordance with the timing of reception,and may, if QOL factor information not satisfying the predeterminedcondition is received, perform the evaluation not in accordance with thetiming of reception (perform the evaluation later).

After performing the evaluation, the server 3 provides the user with anetwork service in accordance with the evaluation result (step S37 shownin FIG. 11, steps S8 and S9 shown in FIG. 4). More specifically, theserver 3 specifies service content to be provided to the user on thebasis of the various types of indices calculated at step S36 (see FIG.10). As described above, in the present embodiment, as the networkservice, (a) presenting the evaluation result, (b) presenting adviceinformation for improving the evaluation result, (c) presentingrecommendation information that recommends a commodity or the like forimproving the evaluation result, (d) providing content for improving theevaluation result, and (e) giving a privilege or the like in accordancewith the evaluation result, are performed.

In the present embodiment, as the service of presenting the evaluationresult, information including an index number representing each of theindices described above is transmitted to the hand-held terminal 5 andpresented to the user (FIG. 24, etc.). As the advice information, forexample, when an evaluation result (environmental index) indicating thatthe user has been in a hot environment is calculated, advice informationshowing a method for preventing exhaustion due to the summer heat may bepresented to the user. As the recommendation information, for example,when an evaluation result (health index) indicating accumulation offatigue is calculated, recommendation information that recommends acommodity that is effective for the recovery from fatigue may bepresented to the user. As content provided by the network service, forexample, when an evaluation result (health index) indicatinginsufficient sleep is calculated, music content that is effective foreasy sleep onset may be provided, and, when an evaluation result(emotion index) indicating depressed feelings is calculated, content forinvigorating the feelings may be provided. As the privilege, forexample, when an evaluation (health index) indicating the usersuccessfully maintaining a healthy state is calculated, a privilege thatgives points that can be used in the network service may be given, or aprivilege of discounting usage charge of the hand-held terminal 5 may begiven.

In the present embodiment, the server 3 specifies the service content onthe basis of the preference information (see FIG. 10). For example, theserver 3 may alter the type of the commodity or the like to berecommended in the recommendation information, on the basis of the genrethat is of interest of the user and indicated by the preferenceinformation. Specifically, even when the same evaluation result iscalculated for a certain user, if the preference information indicatesinterest in physical exercise, the server 3 may present recommendationinformation related to sporting goods, and, if the preferenceinformation indicates in interest in meal, the server 3 may presentrecommendation information related to health food. For example, theserver 3 may alter content of the privilege to be given on the basis ofthe preference information. For example, when a commodity is to begifted to the user as the privilege, the server 3 may determine the typeof the commodity on the basis of the genre of interest of the userindicated by the preference information.

After specifying the service content, the server 3 transmits, to thehand-held terminal 5, the service data associated with the networkservice (see FIG. 11). Examples of the service data include data of theevaluation result, data of advice information, data of recommendationinformation, data of content to be provided, and/or data related toprivilege (data indicating notification of privilege bestowal, data ofthe privilege itself), etc.

Although details will be described later, in the present embodiment,service data associated with the network service in accordance with theevaluation result related to a certain user is in some cases transmittedto a terminal of a user other than the user (e.g., family or friend ofthe user) in addition to the user (person himself/herself) who is thesubject of evaluation. At this moment, the content of the service datatransmitted to the user who is the subject of evaluation and the contentof the service data transmitted to the other user may be different.

As described above, in the present embodiment, an evaluation isperformed by the server 3 at a timing when a predetermined clock timehas arrived, and providing of the network service is performed inresponse to performing of the evaluation. Here, the timing for providingthe network service may be any timing. For example, in anotherembodiment, the timing is not limit to when the predetermined clock timehas arrived, and the network service may be provided at a timing set onthe basis of activity (the activity information) of the user. Forexample, the server 3 may provide the network service at a timing whenthe user leaves office or a timing when the user has returned home.

In another embodiment, when the server 3 performs the evaluation everytime the QOL factor information is received, the server 3 may providethe network service every time the evaluation is performed. In addition,when an evaluation result satisfying the predetermined condition iscalculated, the server 3 may provide the network service based on theevaluation in accordance with the calculated timing, and, when anevaluation result not satisfying the predetermined condition iscalculated, the server 3 may provide (provide later) the network servicenot in accordance with the calculated timing. It should be noted that“when an evaluation result satisfying the predetermined condition iscalculated” is, for example, when an evaluation result based on apredetermined type of QOL factor information is calculated, or when anevaluation result of a predetermined result is calculated. For example,in the example shown in FIG. 10, when the result of an evaluationperformed in response to the end of the meeting indicates that the useris depressed, advice information such as, for example, “take a deepbreath” may be transmitted from the server 3 to the hand-held terminal 5at a timing immediately after the end of the meeting. In addition, forexample, when the result of an evaluation performed in response to theend of movement indicates that the user has moved under a hotenvironment, advice information for prevent heatstroke may betransmitted from the server 3 to the hand-held terminal 5 at a timingimmediately after the end of the movement.

The hand-held terminal 5 receives the service data transmitted from theserver 3. Then, the hand-held terminal 5 presents the user withinformation associated with the service data by using the receivedservice data (step S38). With this, for example, an evaluation result,advice information, and/or recommendation information are displayed onthe display 17 of the hand-held terminal 5, content associated with theservice is reproduced on the hand-held terminal 5, or a notificationindicating that a privilege associated with a service has been given isdisplayed on the hand-held terminal 5.

In another embodiment, the terminal capable of receiving the networkservice does not necessarily have to be the hand-held terminal 5. Morespecifically, the information associated with the service data may bepresented on another device not the hand-held terminal 5. For example,the content to be provided by the network service may be acquired by thehand-held terminal 5 and reproduced by the base device 6. In addition,for example, the server 3 may transmit the service data to apreregistered terminal (e.g., user's home personal computer) differentfrom the hand-held terminal 5.

As described above, in the present embodiment, in the waking period ofthe user, the hand-held terminal 5 calculates the QOL factor informationsuch as the activity information, the environmental information, theemotion information, and the preference information, and transmits theQOL factor information to the server 3. Based on QOL factor informationreceived during the sleeping period and QOL factor information acquiredin the waking period, the server 3 performs an evaluation regarding thehealth, activity, environment, and/or emotion of the user. Then thenetwork service in accordance with the evaluation result is provided tothe user.

As described above, the user mounts the hand-held terminal 5 on the basedevice 6 when going to bed (see FIG. 11). In response, the operationmode of the hand-held terminal 5 is transitioned to the OFF-mode. Inresponse to the operation mode changing to the OFF-mode, the hand-heldterminal 5 ends the process for calculating the QOL factor informationto be calculated during the waking period. More specifically, thehand-held terminal 5 ends the sensing of the positional information bythe position sensing section 13 and sensing of the environment sensorinformation by the environment sensor 14 (step S38 shown in FIG. 11).

In the present embodiment, the hand-held terminal 5 acquires thepositional information and the environment sensor information throughouta single day (during the waking period). Here, in another embodiment,the condition (period and/or timing) for acquiring information forcalculating the QOL factor information may be any condition. Thus,acquiring of the information for calculating the QOL factor informationmay be initiated and/or ended in accordance with satisfying of thepredetermined condition. For example, in another embodiment, thehand-held terminal 5 may end acquiring of the positional information onthe basis of activity (activity information) of the user. Specifically,the hand-held terminal 5 may end acquiring of the positional informationin response to the user being determined to have returned home. Forexample, the hand-held terminal 5 may control initiation and/or end ofacquiring of the environmental information on the basis of activity(activity information) of the user. Specifically, the hand-held terminal5 may acquire the environmental information only while the user isconducting a predetermined activity (e.g., moving, working). Forexample, in another embodiment, an acquiring process may end at a timingwhen the service data is received from the server 3.

In the present embodiment, even after the service data is received,since the positional information and the environment sensor informationare acquired, the activity information and the environmental informationcan be calculated. The hand-held terminal 5 transmits, to the server 3,positional information and environmental information calculated afterreceiving the service data, similarly to before receiving the servicedata. Thus, at the next evaluation timing (timing of evaluation to beperformed tomorrow), the server 3 may perform an evaluation on the basisof positional information and environmental information calculated afterreceiving the service data. In addition, while the user is in bed, theserver 3 may perform an evaluation on the basis of positionalinformation and environmental information calculated after receiving theservice data, and may transmit the service data based on the evaluationto the hand-held terminal 5 at a predetermined timing (e.g., a timingwhen the user awakens in the morning on the next day). For example, byhaving the hand-held terminal 5 perform the notification to the server 3in response to mounting of the hand-held terminal 5 on the base device6, the hand-held terminal 5 can notify the server 3 about when the userhas gone to bed.

In another embodiment, after receiving the service data, the hand-heldterminal 5 may shut down sensing of the positional information and theenvironment sensor information, or may shut down calculation of theactivity information and the environmental information. In a case wherethe service data is received at a rate of once per a predeterminedperiod of time, when the service data is received during thepredetermined period of time, the hand-held terminal 5 may shut downcalculation of the QOL factor information during the predeterminedperiod of time and/or shut down acquiring (sensing) of the informationfor calculating the QOL factor information.

(4-2: Process on Hand-Held Terminal)

Next, specific examples of processes executed on the hand-held terminal5 during the waking period will be described. In the ON-mode (during thewaking period), the hand-held terminal 5 executes an activityinformation calculation process, an environmental informationcalculation process, an emotion information calculation process, apreference information calculation process, and an informationpresentation process. In the present embodiment, these processes areeach executed by different programs (applications). The hand-heldterminal 5 can execute various types of applications throughmultitasking. Other than the processes described above, applicationsdepending on functions of the hand-held terminal 5 which is amultifunctional device may be executed as appropriate. In the following,details of each of the processes described above will be described.

(Activity Information Calculation Process)

FIG. 12 is a flowchart showing one example of the flow of the activityinformation calculation process. The activity information calculationprocess is a process of calculating the activity information indicatingthe activity of the user on the basis of the position (positionalinformation) of the user. In the present embodiment, the activityinformation calculation process is continuously executed in the ON-mode.More specifically, in response to the operation mode of the hand-heldterminal 5 being set to the ON-mode, the CPU of the processing section11 initiates execution of a program for the activity informationcalculation process, and continuously executes the activity informationcalculation process until the operation mode is set to the OFF-mode.

First, at step S51, the processing section 11 acquires information usedfor calculating the activity information. In the present embodiment, theinformation used for calculating the activity information is thepositional information described above. More specifically, theprocessing section 11 acquires the positional information from theposition sensing section 13. In the present embodiment, the processingsection 11 acquires, as clock time information corresponding to thepositional information, clock time information indicating the clock timeat the point in time when the positional information has been acquired.The acquired positional information and clock time information arestored in a memory of the processing section 11.

At step S52, the processing section 11 judges whether or not a conditionto calculate the activity information has been satisfied. Although anycondition may be used as this calculation condition, in the presentembodiment, the user ending his/her movement (first condition), and theuser staying at a location where the activity information is to becalculated (second condition) are used. When either one of the firstcondition and the second condition is satisfied, the condition tocalculate the activity information is judged to be satisfied.

Whether or not the first condition is satisfied can be judged bydetermining, for example, whether or not a sensed position indicated bythe positional information has moved from a certain location and thenstopped at another location for a predetermined time period or longer(e.g., ten minutes or longer). It should be noted that when the sensedposition is not separated from the certain location by a predetermineddistance or more, or the duration of the movement is equal to or shorterthan a predetermined time period, the processing section 11 may assessthat the sensed position has not moved from the certain location. As aresult, through the processes at steps S22, S29, and S33 describedabove, the processing section 11 can determine that the user has movedfrom a certain location to another location (e.g., movement from home toworkplace, movement from workplace to gymnasium, etc.).

Whether or not the second condition is satisfied is judged in the mannerdescribed next. That is, the processing section 11 first judges whetheror not the sensed position has stayed at a certain location for apredetermined time period or longer (e.g., fifteen minutes or longer)and then started moving. When the processing section 11 judges thatstaying for a predetermined time period or longer has not occurred, orthat the movement has not started, the processing section 11 judges thatthe second condition is not satisfied. On the other hand, when theprocessing section 11 judges that movement has started after stayingoccurred, the processing section 11 specifies the type of the stayinglocation. The type of the staying location may be, for example, alocation related to the user such as home or workplace, or a facilitysuch as a gymnasium, a restaurant, or a movie theater. In the presentembodiment, the position of the location related to the user isregistered and stored in the hand-held terminal 5 in advance. Inaddition, the position of a facility is specified by using, for example,map information in which a position on a map and a facility existing atthat position are associated. The map information may be stored in thehand-held terminal 5 in advance, or may be acquired on the hand-heldterminal 5 from an external device via the Internet or the like. Theprocessing section 11 specifies the type of the staying location byspecifying the position matching the position of the staying location,from among the positions of locations related to the user and positionsof the facilities. For example, at step S27 described above, the stayinglocation is specified as the workplace, and, at step S31 describedabove, the staying location is specified as the gymnasium. When the typeof the staying location cannot be specified, the processing section 11judges that the second condition is not satisfied.

Next, the processing section 11 judges whether or not the specifiedstaying location is a location where the activity information is to becalculated. Here, the hand-held terminal 5 stores, in advance, the typesof locations where the activity information is to be calculated. Forexample, in the present embodiment, some locations including theworkplace are stored, but the gymnasium is not stored. The abovedescribed judgment is performed for determining whether or not the typeof the specified staying location matches any of the types of locationsstored in advance. When the specified staying location is judged to be alocation where the activity information is to be calculated, theprocessing section 11 judges that the second condition is satisfied. Onthe other hand, when the specified staying location is judged not to bea location where the activity information is to be calculated, theprocessing section 11 judges that the second condition is not satisfied.For example, at step S27 described above, the workplace that is thestaying location is judged to be a location where the activityinformation is to be calculated, and the second condition is judged tobe satisfied. On the other hand, at step S31 described above, thegymnasium that is the staying location is judged not to be a locationwhere the activity information is to be calculated, and the secondcondition is judged not to be satisfied. In another embodiment, thegymnasium may be registered as a location where the activity informationis to be calculated such that the activity information is calculated inresponse to the user staying at the gymnasium.

As described above, in the present embodiment, the judgment of whetheror not to calculate the activity information is performed on the basisof the positional information (positional information acquired at stepS51 and stored in the memory). Here, in another embodiment, the judgmentmay be performed on the basis of other information. For example, inanother embodiment, the judgment may be performed on the basis of theschedule information of the user. For example, when an end time of workis registered in the schedule information, the hand-held terminal 5 mayjudge to calculate the activity information in response to arrival ofthe end time.

When the judgment result at step S52 is positive, the process at stepS53 is executed. On the other hand, when the judgment result at step S52is negative, the process at step S51 is executed once again. Morespecifically, until the condition is judged to be satisfied at step S52,the processing section 11 repeats the processes at steps S51 and S52. Itshould be noted that the process loop of steps S51 and S52 is repeatedlyexecuted at a rate of, for example, once every predetermined timeperiod.

At step S53, the processing section 11 calculates the activityinformation. FIG. 13 shows one example of the activity informationcalculated in the present embodiment. FIG. 13 shows a series of activityinformation calculated when the user conducts the behaviors shown inFIG. 10. As shown in FIG. 13, in the present embodiment, the activityinformation includes activity content and activity level. In addition,as described above, in the present embodiment, the processing section 11can calculate activity information indicating movement and activityinformation indicating an activity (e.g., work) in accordance with thelocation.

Here, when the first condition is satisfied at step S52, the processingsection 11 calculates the activity information indicating movement(activity information of (a), (c), and (d) in FIG. 13). In the presentembodiment, the activity information indicating movement shows amovement method as the activity content, and a movement amount as theactivity level. More specifically, the processing section 11 calculatesa movement speed and a movement pathway from the positional informationsensed during the movement, and specifies the movement method on thebasis of the movement speed and the movement pathway. For example, whenthe movement speed is lower than a predetermined speed, the movementmethod is specified as walking. In addition, when the movement speed isequal to or higher than the predetermined speed and the movement pathwayis on a road, the movement method is specified as automobile(motorbike). Furthermore, when the movement speed is equal to or higherthan the predetermined speed and the movement pathway is on a railroadtrack, the movement method is specified as train.

It should be noted that when the movement speed changes during movement,multiple types of movement methods may be specified. For example, whenthe movement speed is lower than the predetermined speed at thebeginning of the movement and changes to equal to or higher than thepredetermined speed part way through, the movement method may bespecified as walking and automobile (or train). In this case, theprocessing section 11 may calculate two types of activity information,i.e., activity information in which the movement method is walking andactivity information in which the movement method is automobile (ortrain).

In addition, the processing section 11 can calculate, as the movementamount, moving hours based on clock time information corresponding tothe positional information acquired during movement. In anotherembodiment, the processing section 11 may calculate, as the movementamount, a moved distance on the basis of the movement pathway.

When the second condition is satisfied at step S52, the activityinformation (activity information of (b) in FIG. 13) indicating theactivity in accordance with the location is calculated. “Activity inaccordance with the location” is an activity that can be estimated fromthe staying location of the user, and examples thereof include “work”when the staying location is the workplace and “meal” when the stayinglocation is a restaurant.

As shown in FIG. 13, similarly to the activity information indicatingmovement, the activity information indicating activity in accordancewith the location also includes activity content and activity level(active hours). The activity content is calculated on the basis of thestaying location. Specifically, the hand-held terminal 5 stores a tablein which staying location (e.g., workplace) and the activity content(e.g., work) are associated, and specifies the activity content by usingthe table. Thus, for example, in a case where “workplace” as the stayinglocation and “work” as the activity content are associated in the table,when the staying location is specified as workplace at step S27described above, the activity content is specified as work.

The active hours can be calculated as, for example, the duration of stayat the staying location. This duration can be calculated on the basis ofclock time information corresponding to the positional informationacquired while staying at the staying location.

In the manner described above, the activity information is calculated atstep S53. In a single process at step S53, a single set of activityinformation is calculated and stored in the memory of the processingsection 11.

At step S54, the processing section 11 transmits, to the server 3, theactivity information calculated at step S53. More specifically, theprocessing section 11 transmits the activity information stored in thememory to the server 3 by the communication section 10.

After step S54, the processing section 11 executes the process at stepS51 once again. Subsequently, until the operation mode is altered to theOFF-mode, the processing section 11 repeatedly executes the processes atsteps S51 to S54. As a result, the activity information is calculated ata predetermined timing (e.g., steps S22, S27, S29, and S33 shown in FIG.11) in a single day, and is transmitted to the server 3.

(Environmental Information Calculation Process)

FIG. 14 is a flowchart showing one example of the flow of theenvironmental information calculation process. The environmentalinformation calculation process is a process of calculating theenvironmental information indicating the environment surrounding theuser on the basis of the environment sensor information (ambienttemperature and humidity). In the present embodiment, the environmentalinformation calculation process is continuously executed in the ON-mode.More specifically, in response to the operation mode of the hand-heldterminal 5 being set to the ON-mode, the CPU of the processing section11 initiates execution of a program for the environmental informationcalculation process, and continuously executes the environmentalinformation calculation process until the operation mode is set to theOFF-mode.

First, at step S61, the processing section 11 acquires the environmentsensor information as the information used for calculating theenvironmental information. More specifically, the processing section 11acquires, as the environment sensor information from the environmentsensor 14, temperature information sensed by the temperature sensor, andhumidity information sensed by the humidity sensor. The acquiredenvironment sensor information is stored in the memory of the processingsection 11.

At step S62, the processing section 11 acquires positional informationas information for performing a judgment of whether or not to calculatethe environmental information. The process at step S62 is similar to theprocess at step S51 described above for acquiring the positionalinformation. Thus, the processing section 11 may omit step S62, and usethe positional information acquired at step S51 and stored in thememory.

At step S63, the processing section 11 judges whether or not a conditionto calculate the environmental information has been satisfied. Asdescribed above, in the present embodiment, the environmentalinformation is calculated at a timing when the activity information iscalculated (see steps S22, S27, S29, and S33 shown in FIG. 11). Thus,the judgment at step S63 can be performed depending on whether or notthe activity information has been calculated. When the judgment resultat step S63 is positive, the process at step S64 is executed. On theother hand, when the judgment result at step S63 is negative, theprocess at step S61 is executed once again. More specifically, until thecondition is judged to be satisfied at step S63, the processing section11 repeats the processes at steps S61 to S63. It should be noted thatthe process loop of steps S61 to S63 is repeatedly executed at a rateof, for example, once every predetermined time period (possibly at arate identical to that for the process loop of steps S51 and S52).

At step S64, the processing section 11 calculates the environmentalinformation based on the environment sensor information calculated atstep S61. FIG. 15 shows one example of the environmental informationcalculated in the present embodiment. FIG. 15 shows a series ofenvironmental information calculated when the user conducts thebehaviors shown in FIG. 10. As shown in FIG. 15, in the presentembodiment, the environmental information includes activity content andenvironmental values.

The activity content included in the environmental information isidentical to the activity content included in the activity informationcorresponding to the environmental information (see FIG. 15). In thepresent embodiment, the environmental information indicating theenvironment in which the user performs the activity indicated by theactivity information is calculated. In other words, the activity contentincluded in the environmental information indicates the activity of theuser in a period in which the environment sensor information is sensed.At step S64, the processing section 11 sets the activity contentcalculated at step S53 as the activity content of the environmentalinformation.

The environmental values included in the environmental information showvalues calculated from the environment sensor information acquired inthe active period related to the activity content associated with theenvironmental values. In the present embodiment, the environmentalvalues are each an average value of each of the indices (ambienttemperature and humidity) indicated by the environment sensorinformation in the active period (see FIG. 15). More specifically, theprocessing section 11 first specifies the active period on the basis ofthe activity information. The active period can be specified on thebasis of clock time information of a time point when the activityindicated by the activity information has started, and clock timeinformation at a time point when the activity has ended. Next, theprocessing section 11 reads, from the memory, multiple sets of theenvironment sensor information acquired during the active period,calculates an average value of ambient temperature included in each setof the environment sensor information, and calculates an average valueof humidity included in each set of the environment sensor information.The environmental values are not limited to averages of ambienttemperature and humidity, and may be any value calculated from theenvironment sensor information. For example, if a heat index number (WetBulb Globe Temperature (WBGT)) can be calculated from the environmentsensor information, the heat index number may be calculated as anenvironmental value.

In another embodiment, the environmental information may include historyinformation of each set of the environment sensor information acquiredduring the active period instead the environmental values (or inaddition to the environmental values). The history information mayinclude all the sets of the environment sensor information acquiredduring the active period, or may include one portion of the sets of theenvironment sensor information (e.g., environment sensor informationculled to represent one set per predetermined time period). When theenvironmental information includes the history information, the server 3can recognize a change in the environment (temperature, humidity, etc.)during the active period. In addition, when the evaluation result basedon the environmental information is to be presented to the user, theserver 3 may present the user with the change in the environment duringthe active period. For example, when the evaluation result (e.g., imageshown in FIG. 25) is displayed on the hand-held terminal 5, informationbased on the history information (e.g., line graph representing thehistory information) may be displayed on the hand-held terminal 5 inresponse to a predetermined instruction given by the user (e.g., inresponse to an input is made with respect to a detail button 56 shown inFIG. 25).

The environmental information may be any information representing theenvironment surrounding the user. More specifically, in anotherembodiment, the hand-held terminal 5 may acquire weather informationregarding the weather and include that in the environmental information.The weather information may be acquired from, for example, an externaldevice via the Internet. In addition, for example the hand-held terminal5 may include an air pressure sensor, and may generate environmentalinformation on the basis of air pressure information sensed by the airpressure sensor. The hand-held terminal 5 may include an illuminationsensor, and may generate environmental information on the basis of theillumination information sensed by the illumination sensor. Furthermore,for example, the hand-held terminal 5 may sense noise by the microphone,and may generate environmental information on the basis of the sensednoise. In such manner, the environmental information may indicate atleast one of ambient temperature, humidity, weather, atmosphericpressure, brightness, and noise of the location where the user ispresent.

In the manner described above, the environmental information includingthe activity content and the environmental values is generated at stepS64. As a result, for example, environmental information (environmentalinformation of (a), (c), and (d) shown in FIG. 15) indicating theenvironment while the user is moving and environmental information ((b)shown in FIG. 15) indicating the environment of a period in which theuser is performing a specific activity (e.g., work) are calculated. In asingle process at step S64, a single set of the environmentalinformation is calculated and stored in the memory of the processingsection 11.

Although details will be described later, in the present embodiment, theactivity information is used when calculating an environmental indexbased on the environmental information on the server 3. Thus, in thepresent embodiment, for the purpose of calculating the environmentalinformation in addition to calculating the activity information, theprocessing section 11 calculates the environmental information under acondition that the activity information has been calculated (step S63).Furthermore, the processing section 11 calculates the environmentalinformation on the basis of the environment sensor information acquiredduring the active period corresponding to the activity information (stepS64).

In another embodiment, the condition to calculate the environmentalinformation may be any condition, and is not limited to a conditionrelated to the activity information. In addition, the period in whichthe environmental information is to be calculated may be any period, andis not limited to the active period. For example, in another embodiment,the hand-held terminal 5 may calculate the environmental information ina period from a predetermined start time to a predetermined end time (onthe basis of the environment sensor information acquired during thisperiod), under a condition that the end time has arrived.

In another embodiment, the hand-held terminal 5 may judge whether or notto calculate the environmental information on the basis of thepositional information sensed by the position sensing section 13. Forexample, the hand-held terminal 5 may determine the positionalinformation on the basis of the user leaving and returning home, andcalculate the environmental information for a home-away period (periodbetween leaving home and returning home). In this case, theenvironmental information may be calculated under a condition that theuser has returned home.

In another embodiment, the hand-held terminal 5 may judge whether or notto calculate the environmental information on the basis of the scheduleinformation of the user. For example, when the start time and end timeof work are registered in the schedule information, the hand-heldterminal 5 may calculate the environmental information with respect to aperiod from the start time to the end time. In this case, theenvironmental information may be calculated under a condition that theend time has arrived.

At step S65, the processing section 11 transmits the environmentalinformation calculated at step S64 described above to the server 3. Morespecifically, the processing section 11 transmits the environmentalinformation stored in the memory to the server 3 by the communicationsection 10.

After step S65, the processing section 11 executes the process at stepS61 once again. Subsequently, until the operation mode is altered to theOFF-mode, the processing section 11 repeatedly executes the processes atsteps S61 to S65. As a result, the environmental information iscalculated at a predetermined timing (e.g., steps S22, S27, S29, and S33shown in FIG. 10) in a single day, and is transmitted to the server 3.

(Emotion Information Calculation Process)

FIG. 16 is a flowchart showing one example of the flow of the emotioninformation calculation process. The emotion information calculationprocess is a process of calculating the emotion information indicatingthe emotion of the user on the basis of voice around the hand-heldterminal 5 (voice of the user himself/herself and/or voice of aconversation partner) or facial expression of the user. In the presentembodiment, the emotion information calculation process shown in thefollowing is continuously executed in the ON-mode of the hand-heldterminal 5. More specifically, in response to the operation mode of thehand-held terminal 5 being set to the ON-mode, the CPU of the processingsection 11 initiates execution of a program for the emotion informationcalculation process, and continuously executes the emotion informationcalculation process until the operation mode is set to the OFF-mode.

As a result, in the present embodiment, the emotion information issometimes calculated also in a standby state of the hand-held terminal 5(a state in which the display 17 of the hand-held terminal 5 is turnedOFF). In addition, also while the user is operating the hand-heldterminal 5 and an application other than the program for the emotioninformation calculation process is executed (in the background of theother application), the emotion information is calculated.

First, at step S71, the processing section 11 acquires information forjudging whether or not to initiate measurement of information used forcalculating the emotion information. In the present embodiment, for thepurpose of calculating the emotion information, sound sensed by themicrophone 15 and image captured by the camera 16 are used. However, thesensing by the microphone 15 and the camera 16 sometimes cannot beperformed properly depending on the status of the hand-held terminal 5,and the emotion cannot be calculated correctly in some cases. Forexample, in a situation where the hand-held terminal 5 is placed in abag or a pocket of the user, voice or the like of the user is possiblynot sensed properly by the microphone 15 because of more noises.Furthermore, in a situation where the user is not emitting any voices,the information used for calculating the emotion information cannot beobtained. Furthermore, in a situation where the hand-held terminal 5 isplaced in a bag or pocket of the user or placed on a desk, image of theface of the user conceivably cannot be captured by the camera 16.

Thus, in the present embodiment, the hand-held terminal 5 performs themeasurement by the microphone 15 and the camera 16 in an appropriatesituation (in a situation where the emotion information can becalculated with a certain degree of accuracy). More specifically, thesensing by the microphone 15 and the image capturing by the camera 16are not constantly executed in the ON-mode, and are initiated inresponse to satisfying of a predetermined measuring condition. Theprocess at step S71 is a process of acquiring information for judgingthis measuring condition. Specifically, the processing section 11acquires the schedule information described above and operationinformation indicating an operation with respect to the hand-heldterminal 5. The schedule information is acquired from the storagesection (memory or the like) of the hand-held terminal 5. The operationinformation is acquired from the operation input section 12.

At step S72, the processing section 11 judges whether or not themeasuring condition is satisfied. In the present embodiment, themeasuring condition is satisfying of at least one of a condition(microphone condition) for initiating measurement by the microphone 15and a condition (camera condition) for initiating measurement by thecamera 16. The microphone condition is, for example, initiation of apredetermined event. The predetermined event may be, other than“meeting” described above, for example, meal, work, movement, or thelike. Here, one or more types of events including “meeting” arepredetermined as the predetermined event. In the present embodiment, themicrophone condition is judged on the basis of the schedule information.More specifically, the processing section 11 refers to the scheduleinformation acquired at step S71, and judges whether or not a clock timefor starting any one of the predetermined events determined in advancehas arrived. The processing section 11 judges that the microphonecondition is satisfied when a clock time for starting an event hasarrived, and judges that the microphone condition is not satisfied whena clock time for starting an event has not arrived.

As described above, in the present embodiment, the judgment of whetheror not a predetermined event has started is performed on the basis ofthe schedule information. In another embodiment, the judgment may beperformed on the basis of other information. For example, in anotherembodiment, the processing section 11 may perform the judgment on thebasis of the positional information (and/or the activity informationcalculated from the positional information). For example, the processingsection 11 may judge that a “meal” event has started, when theprocessing section 11 determines that the user has entered a restauranton the basis of the positional information, or when activity informationindicating “meal” is calculated.

In another embodiment, the hand-held terminal 5 may perform measurementby the microphone 15 in a period in which the user is making a telephonecall by using the hand-held terminal 5. More specifically, theprocessing section 11 may use, as the microphone condition, the start ofa telephone call on the hand-held terminal 5. At this moment, at stepS76 described later, the processing section 11 may judge to end themeasurement under a condition that the telephone call has ended on thehand-held terminal 5. With this, the hand-held terminal 5 can detect thevoice of the user in a period in which the user is making a telephonecall on the hand-held terminal 5, and can determine the emotion in thisperiod.

In addition, the camera condition is the start of an operation withrespect to the hand-held terminal 5 by the user. The camera condition isjudged on the basis of the operation information. More specifically, theprocessing section 11 judges whether or not any operation has beenexecuted with respect to the hand-held terminal 5, on the basis of theoperation information. The processing section 11 judges that the cameracondition is satisfied when some operation is executed with respect tothe hand-held terminal 5, and judges that the camera condition is notsatisfied when an operation is not executed with respect to thehand-held terminal 5. It should be noted that the camera condition maybe any condition enabling determination of the user viewing thehand-held terminal 5 (in other words, the user being included in animage capturing range of the camera). For example, in anotherembodiment, the camera condition may be the start of a predeterminedinformation processing (e.g., video reproduction process) on thehand-held terminal 5. At this moment, at step S76 described later, theprocessing section 11 may judge to end the measurement under a conditionthat reproduction of a video has ended.

In another embodiment, the processing section 11 may prevents acondition from being determined as satisfied, in response to anincorrect operation (e.g., when the user not intendedly touches a buttonon the hand-held terminal 5) and/or an operation for a short period oftime (e.g., when the user performs an operation on the hand-heldterminal 5 to simple see the current time). More specifically, theprocessing section 11 may judge that the camera condition is satisfiedwhen an operation is performed for a predetermined number of times ormore in a predetermined period, and may judge that the camera conditionis not satisfied when an operation has not been performed for apredetermined number of times for a predetermined period.

When the judgment result at step S72 is positive, the process at stepS73 is executed. On the other hand, when the judgment result at step S72is negative, the process at step S71 is executed once again. Morespecifically, until the condition is judged to be satisfied at step S72,the processing section 11 repeats the processes at steps S71 and S72. Itshould be noted that the process loop of steps S71 and S72 is repeatedlyexecuted at a rate of, for example, once every predetermined timeperiod.

At step S73, the processing section 11 initiates measurement forcalculating the emotion information. More specifically, At step S72,when the microphone condition is satisfied, the processing section 11turns ON the microphone 15, and initiates sensing by the microphone 15.Furthermore, when the camera condition is satisfied, the processingsection 11 turns ON the camera 16, and initiates sensing (imagecapturing) by the camera 16.

At step S74, the processing section 11 acquires information forcalculating the emotion information. More specifically, when themeasurement by the microphone 15 is being performed, the processingsection 11 acquires sound data sensed by the microphone 15, and storesthe sound data in the memory. In addition, when measurement by thecamera 16 is being performed, the processing section 11 acquires data ofimage captured by the camera 16, and stores the data in the memory. Inthe present embodiment, the sound data or image (video) data acquired inthe measurement period (a period from the start of the process at stepS74 to the end of the measurement at step S77 described later) isaccumulated in the memory, and the emotion information is calculated byusing the accumulated data after the measurement period has ended (stepS77 described later).

At step S75, the processing section 11 acquires information for judgingwhether or not to end the measurement of the information used forcalculating the emotion information. More specifically, when sensing bythe microphone 15 is being performed, the processing section 11 acquiresthe schedule information. Furthermore, when sensing by the camera 16 isperformed, the processing section 11 acquires the operation information.

At step S76, the processing section 11 judges whether or not to end themeasurement. More specifically, when sensing by the microphone 15 isbeing performed, the processing section 11 judges whether or not thepredetermined event, judged to have started at step S72, has ended.Thus, the processing section 11 refers to the schedule informationacquired at step S75, and judges whether or not a clock time when thepredetermined event is to be ended has arrived. The processing section11 judges to end the measurement when the clock time at which the eventis to be ended has arrived. On the other hand, the processing section 11judges not to end (judges to continue) the measurement when the clocktime at which the event is to be ended has not arrived.

As described above, in the present embodiment, judgment of whether ornot to perform sensing by the microphone 15 is conducted depending onwhether or not the current time is in a period of a predetermined event.In another embodiment, the judgment may be performed depending onwhether or not the user is moving. For example, when the hand-heldterminal 5 has a sensor capable of sensing motion (acceleration sensor,etc.), the hand-held terminal 5 judges whether or not the user is inmotion on foot on the basis of the sensing result from the accelerationsensor, and, when the user is in motion on foot, may suspend the sensingby the microphone 15. This is because, when the user is in motion onfoot, the hand-held terminal 5 is envisioned to be placed in a bag or apocket of the user, and sensing the voice of the user accurately isdifficult.

When sensing by the camera 16 is performed, the processing section 11judges whether or not an operation of the user with respect to thehand-held terminal 5 has ended. This judgment is conducted, for example,depending on whether or not an operation has not been performed withrespect to the hand-held terminal 5 in a predetermined time period. Theprocessing section 11 judges to end the measurement when an operationwith respect to the hand-held terminal 5 is judged to have ended. On theother hand, the processing section 11 judges not to end (to continue)the measurement when the operation with respect to the hand-heldterminal 5 is judged not to have ended.

As described above, in the present embodiment, the sensing by the camera16 is performed in a period in which an operation is performed withrespect to the hand-held terminal 5. In another embodiment, the judgmentof whether or not the current time is in the period described above maybe performed on the basis of, not the operation information, but otherinformation. For example, when the hand-held terminal 5 has a functionof turning OFF the display 17 in a period where an operation is notperformed and turning ON the display 17 in response to performing of anoperation, the judgment described above may be performed depending onwhether or not the display 17 is turned ON. Furthermore, the judgmentmay be performed depending on whether or not a predetermined applicationhas been started up on the hand-held terminal 5.

When the judgment result at step S76 is positive, the process at stepS77 is executed. On the other hand, when the judgment result at step S76is negative, the process at step S74 is executed once again. Morespecifically, the processing section 11 repeats the processes at stepsS74 to S76 until the judgment to end the measurement is made at stepS76. It should be noted that the process loop of steps S74 to S76 isrepeatedly executed at a rate of, for example, once every predeterminedtime period.

At step S77, the processing section 11 ends the sensing (measurement) bythe microphone 15 or the camera 16, and calculates the emotioninformation. More specifically, when a sound sensed by the microphone 15is acquired in the process at the immediately preceding step S74, theprocessing section 11 calculates the emotion information by using thesound sensed during the measurement period (i.e., on the basis of voicesof the user and person in the surrounding area). Furthermore, when animage captured by the camera 16 has been acquired in the process at theimmediately preceding step S74, the processing section 11 calculates theemotion information by using the image captured during the measurementperiod (i.e., on the basis of facial expression of the user).

First, one example of the method for calculating the emotion informationby using sound sensed by the microphone 15 will be described. In thepresent embodiment, first, the processing section 11 identifies thevoice of the user and the voice of a person (conversation partner) otherthan the user in signals of sound sensed during the measurement period.The method for identifying voice of the user and a conversation partnermay be a conventional voice identification technology, and may beperformed in the following manner, for example. More specifically, theprocessing section 11 removes a silent segment and a segment in whichdetermination of emotion is difficult (e.g., a segment with many noises,a segment in which multiple voices overlap each other) in signals ofsound measured during the measurement period. Then, the processingsection 11 judges whether or not there is any segment containing thevoice of the user in remaining voice segments. This judgment can beperformed by registering (storing) the voice of the user in thehand-held terminal 5 in advance, and comparing the preregistered uservoice and a voice in the remaining voice segments. For example, theprocessing section 11 may calculate a similarity for predeterminedcharacteristic amounts related to both voices for each unit segment, andassess that the unit segment contains the voice of the user if thesimilarity is not lower than a predetermined threshold or assess thatthe unit segment contains a voice of a conversation partner if thesimilarity is lower than a predetermined threshold. As a result, fromthe signals of sound measured during the measurement period, a segmentcontaining the voice of the user (user voice segment) and a segmentcontaining a voice of the conversation partner (partner voice segment)can be extracted.

Next, the processing section 11 determines the emotion of the user onthe basis of the extracted voice of the user. Any method may be used asthe method for determining the emotion on the basis of the voice of theuser, and a conventional emotion distinguishing method may be used. Inthe present embodiment, for example, the determination of the emotion isperformed by the following method. Specifically, the processing section11 calculates the characteristic amounts for each predetermined unitsegment by using voice signals in the user voice segment. As thecharacteristic amounts, for example, intensity, tempo, and intonationare calculated. The intensity can be calculated as an average of themagnitude of amplitude in a predetermined segment. The tempo can beobtained by calculating the number of phonemes in a unit of time. Theintonation can be obtained by detecting delimiters of a text, anddetecting a specific intensity change pattern for a voice within thedelimiters.

The processing section 11 calculates indices indicating the emotion onthe basis of the calculated characteristic amounts. In the presentembodiment, the processing section 11 calculates an index for each ofthe three types of emotions of anger, joy, and sadness on the basis ofthe characteristic amounts. Since these emotions can be estimated fromchanges in the characteristic amounts, the processing section 11calculates the changes in the characteristic amounts (intensity, tempo,and intonation), and calculates each of the indices on the basis of thechange levels thereof. In the present embodiment, the hand-held terminal5 stores a table showing a change pattern of each of the characteristicamount for each of the types of emotions (anger, joy, and sadness). Thechange pattern may be created from an experiment (i.e., on the basis ofa result of actually sensing voices from multiple test subjects andcalculating characteristic amounts). The processing section 11 comparesa change pattern shown by the table and a change in the calculatedcharacteristic amounts for each of the types of emotions, and calculatesa similarity for each of the types of emotions. Each of the indices iscalculated such that a value of an index becomes high for a type ofemotion with a high similarity and a value of an index becomes low for atype of emotion with a low similarity. For example, indices of “anger:80, joy: 10, sadness: 50” are calculated when the similarity is highbetween a change pattern shown in the table regarding “anger” and thechanges in the calculated characteristic amounts, when the similarity islow between a change pattern shown in the table regarding “joy” and thechanges in the calculated characteristic amounts, and when thesimilarity is at a moderate level between a change pattern shown in thetable regarding “sadness” and the changes in the calculatedcharacteristic amounts. In the manner described above, the indicesindicating the emotion of the user based on the voice of the user arecalculated.

In present embodiment, similarly to the voice of the user, theprocessing section 11 calculates the indices indicating the emotion byusing the voice of the conversation partner. More specifically, theprocessing section 11 calculates the indices indicating the emotion ofthe conversation partner by using the same calculation method describedabove.

Here, from the emotion of the conversation partner, the emotion of theuser himself/herself is possibly estimated. For example, when theconversation partner is angry, since the user is (or can be estimated tobe) scolded, the user may conceivably carry the emotion of “hate(hatred)”. Furthermore, when the conversation partner is joyful, theuser may conceivably carry the emotion of “pleasure”.

Thus, in the present embodiment, the processing section 11 calculatesthe indices indicating the emotion of the user on the basis of theemotion of the conversation partner. Specifically, indices indicating“hate” and “pleasure” of the user are each calculated in accordance witheach of the indices of “anger” and “joy” calculated for the conversationpartner. The index indicating hate of the user is calculated to be largewhen the numerical value of the index indicating anger of theconversation partner is large. Furthermore, the index indicatingpleasure of the user is calculated to be large when the numerical valueof the index indicating joy of the conversation partner is large. In thepresent embodiment, although the type of the emotion of the conversationpartner and the type of emotion of the user calculated therefrom aredifferent, in another embodiment, the type of the emotion of theconversation partner and the type of the emotion of the user calculatedtherefrom may be the same. For example, the processing section 11 maycalculate (correct) the index indicating sadness of the user on thebasis of the index indicating sadness of the conversation partner.

As described above, in the present embodiment, indices indicating fivetypes of emotions of anger, joy, sadness, hate, and pleasure arecalculated for each of the predetermined unit segments which are each aunit in which the characteristic amounts are calculated. The processingsection 11 calculates the emotion information in a single measurementperiod on the basis of each of the indices in each of the unit segments.For example, in the present embodiment, the processing section 11calculates an average value of each of the unit segments for each of thefive types of the indices. The processing section 11 uses informationincluding the average values of the calculated five types of indices asthe emotion information. More specifically, in the present embodiment,the emotion information indicating the five types of the emotion iscalculated.

It should be noted that the specific content of the emotion informationis not limited to those described above, and may be any content. Forexample, in another embodiment, the emotion information may beinformation indicating some of the five types of the indices, or mayindicate a type of emotion different from the five types (e.g.,surprise, fear, affection, etc.). Furthermore, the emotion informationis not limited to information indicating multiple numerical values, andmay be information indicating a single type of numerical value. Forexample, the emotion information may be information indicating a scalarvalue that represents heightening (depression) of emotion. Furthermore,the emotion information may be information indicating two types ofindices, i.e., an index representing the level of positive type emotions(joy, pleasure, etc.), and an index representing the level of negativetype emotions (anger, sadness, etc.).

In another embodiment, the hand-held terminal 5 may determine theemotion of the user on the basis of sound (voice) of the surroundingarea in addition to (or instead of) the voice of the user and the voiceof the conversation partner. More specifically, the hand-held terminal 5may determine the emotion (of the user) in accordance with thesurrounding atmosphere of the user. For example, when the hand-heldterminal 5 senses laughter in the surrounding area of the user, thehand-held terminal 5 may estimate that the user is in a pleasantenvironment and determine that the user is joyful (e.g., calculate theindex indicating joy to be high). Furthermore, when the hand-heldterminal 5 senses screaming or an angry roar in the surrounding area ofthe user, the hand-held terminal 5 may estimate that the user is in atense environment and determine that the user is tense (e.g., calculatean index indicating tension to be high).

Next, one example of the method for calculating the emotion informationby using an image captured by the camera 16 will be described. In thepresent embodiment, since the camera 16 is attached facing inward andcapturing of an image by the camera 16 is performed when the hand-heldterminal 5 is operated, the captured image conceivably contains the faceof the user. First, the processing section 11 specifies somecharacteristic points of the face (e.g., eyes, mouth, eyebrows, andcheek bones) of the user with an image recognition process performedwith respect to the image captured by the camera 16. The characteristicpoints may be calculated from a conventional image recognition process.Of the captured images acquired during the measurement period, theprocessing section 11 calculates each of the characteristic points inone or more captured images (all or some of the captured images thathave been acquired) acquired in a predetermined time interval.

Next, the processing section 11 determines the emotion of the user onthe basis of each of the characteristic points. In the presentembodiment, the emotion is determined on the basis of the position ofeach of the characteristic points and changes in the characteristicpoints. Specifically, by capturing an image of the usual facialexpression of the user by the camera 16 in advance, the hand-heldterminal 5 registers the usual position of each of the characteristicpoints of the user in advance. The processing section 11 calculateschange (change direction and change level) from ordinary times and thespeed of the change for each of the calculated characteristic points.

Here, the hand-held terminal 5 stores in advance a table in which thechange and change speed, and the type of emotion (anger, joy, orsadness) in that case are associated. The table may be created from anexperiment (i.e., on the basis of a result of actually calculating thechange and change speed from an image of the face for each type ofemotions in multiple test subjects). The processing section 11 refers tothe table, and calculates an index representing an emotion in accordancewith the change and change speed of each of the calculatedcharacteristic points. Specifically, a similarity between the change andchange speed of each of the calculated characteristic points and thechange and change speed of each type of emotions shown in the table iscalculated, and an index in accordance with the similarity is calculatedfor each type of emotions. For example, when the similarity for theemotion “anger” is calculated to be high, the index for anger iscalculated to be high. As a result, even when the emotion is to becalculated on the basis of facial expression of the user, for example,indices of “anger: 80, joy: 10, sadness: 50” are calculated for eachtype of emotions, similarly to when calculating the emotion on the basisof voices.

In the manner described above, the three types of indices are calculatedfrom the face of the user recognized from a single captured image. Theprocessing section 11 calculates the emotion information in themeasurement period on the basis of the three types of indices calculatedfor each captured image acquired in the predetermined time interval. Forexample, in the present embodiment, an average value of each of thethree types of indices calculated in each of the captured images iscalculated. The processing section 11 uses, as the emotion information,information including the average values of the three types of indicescalculated in the manner described above. Thus, in the presentembodiment, the emotion information indicating the three types ofemotion is calculated.

As described above, in the present embodiment, when determining theemotion by using the captured image, the emotion information includingindices for three types of emotions of “anger”, “joy”, and “sadness” iscalculated. In another embodiment, the emotion information may indicatesome of the three types of indices, or may indicate a type of emotion(e.g., hate, pleasure, surprise, fear, affection, etc.) different fromthe three types. For example, the five types of indices identical towhen determining the emotion by using voices may be calculated.Similarly to using voices, also when using the captured image, theemotion information is not limited to information indicating multiplenumerical values, and may be information indicating a single type ofnumerical value or information indicating two types of indices, i.e., anindex representing the level of positive type emotions and an indexrepresenting the level of negative type emotions.

At step S78, the processing section 11 transmits the emotion informationto the server 3. More specifically, the processing section 11 transmitsthe emotion information calculated at step S77 and stored in the memoryto the server 3 by the communication section 10.

After step S78, the processing section 11 executes the process at stepS71 once again. Subsequently, until the operation mode is altered to theOFF-mode, the processing section 11 repeatedly executes the processes atsteps S71 to S78. With this, the emotion information is calculated inresponse to performing of a predetermined event of a single day (e.g.,step S25 shown in FIG. 11) and is transmitted to the server 3.

In the present embodiment, after the predetermined event has ended, thehand-held terminal 5 calculates the emotion information indicating theemotion of the user in a period of the event. In another embodiment, thehand-held terminal 5 may successively calculate the emotion informationduring the event. For example, the hand-held terminal 5 may calculatethe emotion information for each of the predetermined unit segmentswhich are each a unit in which the characteristic amounts of the voiceof the user is calculated. At this moment, the hand-held terminal 5 maysuccessively transmit the calculated emotion information to the server3. With this, the hand-held terminal 5 can calculate the emotioninformation in real time, and the server 3 can also provide, in realtime, the user with a network service in accordance with an evaluationbased on the emotion information.

(Other Example of Emotion Information Calculation Process)

Next, another example of the emotion information calculation processwill be described. In the emotion information calculation process thepresent embodiment described above, for the purpose of performing themeasurement by the microphone 15 and the camera 16 under a propersituation, the period in which a predetermined event is performed isspecified on the basis of the schedule information, and measurement isperformed in the specified period (step S72 to S76). In anotherembodiment, the period for sensing (measuring) the information forcalculating the emotion information may be specified on the basis ofother information. For example, the hand-held terminal 5 may, byintermittently performing sensing by using the microphone 15, specifythe period in which the predetermined event is performed. In thefollowing, as another one example of the emotion information calculationprocess, an example in which the sensing by the microphone 15 isperformed intermittently will be described.

FIG. 17 is a flowchart showing another one example of the flow of theemotion information calculation process. Similarly to the process shownin FIG. 16, the process shown in FIG. 17 is continuously executed in theON-mode of the hand-held terminal 5.

First, at step S81, the processing section 11 executes sensing by themicrophone 15 for a predetermined time period. More specifically, theprocessing section 11 turns ON the microphone 15, performs sensing bythe microphone 15 for the predetermined time period (e.g., threeseconds), and then turns OFF the microphone 15. With this, theprocessing section 11 acquires sound for the length of the predeterminedtime period.

At step S82, the processing section 11 judges whether or not a measuringcondition to initiate measurement for calculating the emotioninformation on the basis of the sensed sound acquired at step S81 issatisfied. Specifically, the processing section 11 judges whether or nota sound (e.g., the voice of the user or the voice of the conversationpartner) to be used for calculating the emotion information is containedin the sensed sound. When the sound is contained, the measuringcondition is judged to be satisfied, whereas, when the sound is notcontained, the measuring condition is judged not to be satisfied. Whenthe judgment result at step S82 is negative, the process at step S83 isexecuted. On the other hand, when the judgment result at step S82 ispositive, a process at step S84 described later is executed.

At step S83, the processing section 11 stands by for a predeterminedfirst time period (e.g., ten seconds). When the first time periodelapses, the processing section 11 executes the process at step S81 onceagain. Thus, until the measuring condition is satisfied, the processingsection 11 performs the sensing (step S81) by the microphone at a timeinterval equal to the first time period.

On the other hand, at step S84, the processing section 11 initiates themeasurement for calculating the emotion information. The process at stepS84 is similar to the process at step S73 described above.

At step S85, the processing section 11 acquires information forcalculating the emotion information. More specifically, similarly to theprocess at step S74 when performing the measurement by the microphone15, the processing section 11 acquires data of sound sensed by themicrophone 15, and stores the data in the memory. At step S81 describedabove, although sound is intermittently sensed by the microphone 15,sound is continuously sensed at step S85.

At step S86, the processing section 11 judges whether or not to end themeasurement. In the example shown in FIG. 17, this judgment is performedon the basis of sound sensed at step S85. Specifically, a judgment toend the measurement is made when a sound (the voice of the user or thevoice of the conversation partner) to be used for calculating theemotion information is not contained in sound sensed in the pastpredetermined time period (e.g., one minute) from the present time. Onthe other hand, a judgment to not end the measurement is made when thesound to be used for calculating the emotion information is contained inthe sound sensed in the past predetermined time period from the presenttime. When the judgment result at step S86 is positive, the process atstep S87 is executed. On the other hand, when the judgment result atstep S86 is negative, the process at step S85 is executed once again.Subsequently, the processes at steps S85 to S86 are repeatedly executeduntil the judgment to end the measurement is made at step S86. When theprocess loop of steps S85 to S86 is to be repeatedly executed, theprocessing section 11 acquires the sound sensed by the microphone 15such that the sensed sound is temporally continuous. Thus, as a resultof the process loop, sound is continuously sensed in the measurementperiod. However, in another embodiment, sound may be sensed in themeasurement period intermittently with a time interval equal to a secondtime period that is shorter than the first time period.

At step S87, the processing section 11 calculates the emotioninformation. The process of calculating the emotion information at stepS87 is similar to the process of calculating the emotion information onthe basis of sound sensed by the microphone 15 at step S77 describedabove. However, in the example shown in FIG. 17, the length of themeasurement period (period from the start of the process at step S84 tothe end of the measurement at step S87) may not have a sufficient lengthfor calculating the emotion information. Thus, the processing section 11may skip calculating the emotion information when the length of themeasurement period (in other words, the length of sound sensed in themeasurement period) is shorter than a predetermined time period, andcalculate the emotion information when the length of the measurementperiod is equal to or longer than the predetermined time period. In thefollowing step S88, the processing section 11 transmits the emotioninformation to the server 3 similarly to step S78 described above.

After step S88, the processing section 11 executes the process at stepS81 once again. Subsequently, until the operation mode is altered to theOFF-mode, the processing section 11 repeatedly executes the processes atsteps S81 to S88. With this, measurement by the microphone 15 isexecuted in a period in which information (voice of the user, etc.) forcalculating the emotion information can be sensed in a single day, andthe emotion information is calculated for the period.

In the description above, the method for setting the measurement periodof the microphone 15 in the case of calculating the emotion informationby using the sound sensed by the microphone 15 has been described. Alsoin the case of calculating the emotion information by using an imagecaptured by the camera 16, the measurement period (period for capturingan image) by the camera 16 can be set by using a method similar to themethod shown in FIG. 17. More specifically, the processing section 11acquires an image captured by the camera 16 at steps S81 and S85. Inaddition, at step S82, the processing section 11 judges whether or notthe face of the user is included in a captured image. Furthermore, atstep S86, the processing section 11 judges not to end the measurementwhen the face of the user is included in an image captured in the pastpredetermined time period from the present time, and judges to end themeasurement when the face of the user is included in the captured image.In the manner described above, in the case of calculating the emotioninformation by using an image captured by the camera 16, the measurementperiod can be set in accordance with the period in which the emotioninformation can be calculated.

(Preference Information Calculation Process)

FIG. 18 is a flowchart showing one example of the flow of the preferenceinformation calculation process. The preference information is a processof calculating preference information indicating the preference of theuser on the basis of an input with respect to the hand-held terminal 5by the user or the position (activity) of the user. In the presentembodiment, the preference information calculation process iscontinuously executed in the ON-mode. More specifically, in response tothe operation mode of the hand-held terminal 5 being set to the ON-mode,the CPU of the processing section 11 initiates execution of a programfor the preference information calculation process, and continuouslyexecutes the preference information calculation process until theoperation mode is set to the OFF-mode.

First, at step S91, the processing section 11 acquires information to beused for calculating the preference information. In the presentembodiment, the processing section 11 acquires the positionalinformation sensed by the position sensing section 13 and informationinputted by the user (user input information) with respect to theoperation input section 12. The acquired information is stored in thememory of the processing section 11.

At step S92, the processing section 11 judges whether or not acalculation condition to calculate the preference information issatisfied. The calculation condition includes a condition regardingposition (position condition) and a condition regarding input of theuser (user input condition). When either one of the position conditionand the user input condition is satisfied, the calculation condition isjudged to be satisfied.

In the present embodiment, the position condition is the user staying ata specific location where the preference information is to be calculated(location where the preference of the user can be estimated from thelocation). More specifically, the processing section 11 judges whetheror not the user has stayed at a certain location on the basis of theposition sensed by the position sensing section 13. This judgment can beperformed similarly to the judgment for determining whether or not thesecond condition is satisfied as described in “(Activity InformationCalculation Process)” above. When the user is judged to have stayed at acertain location, the processing section 11 specifies the type ofstaying location, and judges whether or not the preference informationcan be calculated for (the type of) the specified staying location. Asdescribed above, the hand-held terminal 5 stores, in advance, the tableshowing the association between location (facility) and genre indicatedby the preference information. The processing section 11 judges whetheror not the preference information can be calculated depending on whetheror not the specified staying location is included in the table. Whencalculation of the preference information is judged to be possible, theprocessing section 11 judges that the position condition is satisfied.On the other hand, the processing section 11 judges that the positioncondition is not satisfied when (a) the user is judged not to havestayed at a certain location, (b) the specified staying location is notincluded in the table, or (c) the type of the staying location cannot bespecified. For example, at step S31, since the user is determined to bestaying at the gymnasium and the gymnasium is included in the table, theposition condition is judged to be satisfied. On the other hand, at stepS27, since the user is determined to be staying at the workplace and theworkplace is not included in the table, the position condition is judgednot to be satisfied.

The user input condition is an input of information from which thepreference can be estimated. Specifically, in the present embodiment,when a specific search phrase with which estimation of the preferenceinformation is possible is inputted in a search engine website, theprocessing section 11 judges that the user input condition is satisfied.More specifically, when an input of a search phrase is included in theinput acquired at step S91, the processing section 11 judges whether ornot the search phrase enables calculation of the preference information,i.e., whether the search phrase is a word that enable estimation of thepreference of the user. The hand-held terminal 5 stores a table showingthe association between words and a genre of preference corresponding tothe words (genre indicated by the preference information). Theprocessing section 11 judges whether or not the inputted search phrasecontains a word described in the table. When the inputted search phrasecontains a word described in the table, the processing section 11assesses that the preference information can be calculated from theinputted search phrase, and judges that the user input condition issatisfied. On the other hand, when the input acquired at step S91 doesnot contain an input of a search phrase, or when the inputted searchphrase does not contain a word described in the table, the processingsection 11 judges that the user input condition is not satisfied.

When the judgment result at step S92 is positive, the process at stepS93 is executed. On the other hand, when the judgment result at step S92is negative, the process at step S91 is executed once again. Morespecifically, the processing section 11 repeats the processes at stepsS91 and S92 until the calculation condition is judged to be satisfied atstep S92. It should be noted that the process loop of steps S91 and S92is repeatedly executed at a rate of, for example, once everypredetermined time period.

At step S93, the processing section 11 calculates the preferenceinformation. When the position condition is satisfied at step S92, theprocessing section 11 calculates the preference information on the basisof the positional information acquired at step S91 (more specifically,on the basis of the staying location calculated from the positionalinformation). That is, the processing section 11 specifies the genreassociated with the staying location specified from the positionalinformation in the table showing the association between location andpreference genre. The processing section 11 calculates the preferenceinformation indicating the specified genre.

When the user input condition is satisfied at step S92, the processingsection 11 calculates the preference information on the basis of theuser input information acquired at step S91. That is, the processingsection 11 specifies the genre associated with the word contained in thesearch phrase specified from the user input information in the tableshowing the association between words and preference genre. Theprocessing section 11 calculates the preference information indicatingthe specified genre.

In the manner described above, preference information is calculated atstep S93. In a single process of step S93, a single set of thepreference information is calculated and stored in the memory of theprocessing section 11.

At step S94, the processing section 11 transmits the preferenceinformation calculated at step S93 to the server 3. More specifically,the processing section 11 transmits, to the server 3 by thecommunication section 10, the preference information stored in thememory.

As described above, in the present embodiment, the preferenceinformation is calculated on the basis of the positional information. Inanother embodiment, the preference information may be calculated on thebasis of the activity information. More specifically, the hand-heldterminal 5 may calculate the activity information (e.g., activityinformation indicating that physical exercise has been conducted at thegymnasium) from positional information, and calculate the preferenceinformation on the basis of the activity information.

In the present embodiment, the hand-held terminal 5 calculates thepreference information on the basis of a search phrase inputted on asearch engine website. In another embodiment, the preference informationmay be calculated on the basis of another user input information. Forexample, when the user performs an input for browsing a web page, thehand-held terminal 5 may determine the preference of the user on thebasis of the content of the web page (e.g., keywords in the web page).For example, when a so-called a “Like” button is pressed on an SNS(social networking service) web page, the hand-held terminal 5 maydetermine the preference of the user on the basis of the content of theweb page.

In another embodiment, the preference information may be calculated onthe basis of information other those described above. For example, thepreference information may be calculated on the basis of profileinformation (age, whether having a child or not, marital status, familycomposition, gender, income, hobby, etc.) of the user stored in thehand-held terminal 5. In addition, the preference information may becalculated on the basis of an application executed on the hand-heldterminal 5. For example, the hand-held terminal 5 can possibly determinethe preference of the user (genre of interest of the user) on the basisof the type of application (e.g., game application, etc.) executed onthe terminal itself.

(Information Presentation Process)

FIG. 19 is a flowchart showing one example of the flow of theinformation presentation process. The information presentation processis a process of presenting (outputting an image and/or sound) the userwith information (information associated with network service) generatedby the server 3. In the present embodiment, the information presentationprocess is continuously executed in the ON-mode. More specifically, inresponse to the operation mode of the hand-held terminal 5 being set tothe ON-mode, the CPU of the processing section 11 initiates execution ofa program for the information presentation process, and continuouslyexecutes the information presentation process until the operation modeis set to the OFF-mode.

First, at step S101, the processing section 11 receives an input fromthe user. More specifically, information indicating an input performedwith respect to the operation input section 12 is acquired from theoperation input section 12. In the present embodiment, the server 3manages a website for providing a network service. The user can, byaccessing this website by using the hand-held terminal 5, browse theevaluation result by the server 3, and make a purchase of a commodity orthe like related to the recommendation information presented by theserver 3. In step S101 described above, the processing section 11receives an input for making various types of requests to the website.Examples of the requests include a request to login to the website, arequest to browse a web page in the website (including a page showingthe evaluation result, and a page for purchasing a commodity or thelike), and a request to purchase a commodity or the like. The programfor executing the information presentation process may be a part of adedicated application for receiving the network service by the server 3,or a part of a widely used browser application for browsing a web page.

At step S102, the processing section 11 executes a process in accordancewith the input received at step S101. For example, when a request oflogin to the website is made, the processing section 11 requests loginto the server 3. For example, when a request to browse a web page in awebsite is made, the processing section 11 requests the server 3 toacquire the web page according to the request. Specifically, theprocessing section 11 causes the communication section 10 to transmitinformation indicating the request to the server 3.

In response to the request described above, the server 3 executes aprocess in accordance with the request (step S119 described later). Forexample, when a request of login is received, the server 3 transmitsinformation of a login image (web page) to the hand-held terminal 5.Furthermore, for example, when a request to acquire a web page isreceived, the server 3 transmits information of the web page to thehand-held terminal 5.

The processes at steps S101 and S102 may be performed with a methodsimilar to a conventional process for making various types of requestswith respect to a server that provides web pages.

At step S103, the processing section 11 judges whether or notinformation related to the network service has been received from theserver 3. The information received from the server 3 in the process atstep S103 is information transmitted from the server 3 in response tothe request at step S102, or information (i.e., service data)transmitted from the server 3 at a timing (step S36 shown in FIG. 11)when the server 3 provides the network service. When the judgment resultat step S103 is positive, the process at step S104 is executed. On theother hand, when the judgment result at step S103 is negative, theprocess at step S101 is executed once again.

At step S104, the processing section 11 presents the user with theinformation received from the server 3. More specifically, on the basisof the information received from the server 3 by the communicationsection 10, the processing section 11 displays an image of a web page orthe like on the display 17, or outputs audio from the loudspeaker 18. Inanother embodiment, the hand-held terminal 5 may include a vibrationmotor as an output device, and, in this case, the service datatransmitted from the server 3 may contain content for controllingvibration of the hand-held terminal 5. Thus, the hand-held terminal 5may outputs image, sound, and/or vibration associated with the servicedata.

Next after step S104, the processing section 11 executes the process atstep S101 once again. Subsequently, until the operation mode is alteredto the OFF-mode, the processing section 11 repeatedly executes theprocesses at steps S101 to S104. With this, the user can browseinformation provided from the server 3 at a timing desired by the userin a single day, and information is provided from the server 3 to thehand-held terminal 5 through “push” at a predetermined timing (step S36shown in FIG. 11) in a single day.

5. Operation on Server

Next, specific examples of processes executed on the server 3 will bedescribed. As described above, the hand-held terminal 5 transmitsvarious types of QOL factor information to the server 3. The server 3evaluates the state (QOL, etc.) of the user on the basis of the QOLfactor information, and provides a network service in accordance withthe evaluation result. In the following, details of the processesexecuted on the server 3 will be described.

FIG. 20 is a flowchart showing one example of the flow of processesexecuted on the server. The flowchart shown in FIG. 20 is continuouslyexecuted regardless of the mode of the hand-held terminal 5 (i.e.,regardless of being in the waking period or the sleeping period).

First, at step S111, the server 3 judges whether or not the QOL factorinformation has been received from the hand-held terminal 5. Asdescribed above, the hand-held terminal 5 transmits the QOL factorinformation to the server 3 at an appropriate timing (step S19 shown inFIG. 4, step S54 shown in FIG. 12, step S65 shown in FIG. 14, step S78shown in FIG. 16, step S94 shown in FIG. 18). In response, the server 3receives the QOL factor information from the hand-held terminal 5, and,at this moment, the judgment result at step S111 becomes positive. Whenthe judgment result at step S111 is positive, the process at step S112is executed. On the other hand, when the judgment result at step S111 isnegative, the process at step S112 is skipped, and the process at stepS113 described later is executed.

At step S112, the server 3 stores (accumulates) QOL factor informationreceived from the hand-held terminal 5 for each user. More specifically,the server 3 updates, so as to include the received QOL factorinformation, the user data indicating various types of informationrelated to the user. The user data stored in the server 3 will bedescribed here.

FIG. 21 shows one example of data organization of the user data storedin a memory on the server of the present embodiment. FIG. 21 shows theuser data stored in the memory of the server 3 for a single user.Although not diagrammatically represented, the server 3 stores, in apredetermined storage section, data similar to the user data shown inFIG. 21 for each user on the network service.

As shown in FIG. 21, the user data contains user identification data,notification user data, user status data, evaluation result data, andprivilege data. In the server 3, a set of these five types of data isaccumulated for each user. It should be noted that the user data maycontain data other these five types of data, or does not necessarilyhave to contain one portion of data among these five types of data.

The user identification data is data indicating identificationinformation (ID, password, etc.) for identifying the user. In thepresent embodiment, in a predetermined registration process executedwhen initiating the network service for the user, the identificationinformation is set and the user identification data indicating the setidentification information is stored in the server 3.

The notification user data is data indicating another user (referred toas “notification-subject user”) to which the evaluation result (orinformation based on that) related to the user indicated by the useridentification data is presented. Specifically, the notification userdata indicates identification information (e.g., ID) of thenotification-subject user. As described above, in the presentembodiment, service data associated with a network service in accordancewith an evaluation result related to a certain user is transmitted to,other than the user (person himself/herself) who is the subject ofevaluation, a terminal of another user. The notification user dataindicates the other user to which the service data is to be transmitted.The notification-subject user does not necessarily have to be a user whocan receive the network service. The notification user data may indicateinformation representing transmission destination to which the servicedata is to be transmitted (e.g., E-mail address or the like of aterminal used by the notification-subject user).

The user status data is data indicating various types of statuses of theuser. As shown in FIG. 21, in the present embodiment, the user statusdata contains health data, activity data, environmental data, emotiondata, and preference data.

The health data is data indicating the health information acquired fromthe hand-held terminal 5 of the user. The activity data is dataindicating the activity information acquired from the hand-held terminal5 of the user. The environmental data is data indicating theenvironmental information acquired from the hand-held terminal 5 of theuser. The emotion data is data indicating the emotion informationacquired from the hand-held terminal 5 of the user. These four types ofdata each include multiple sets of QOL factor information among thepreviously acquired QOL factor information. Specifically, in the presentembodiment, for example, the four types of data indicate QOL factorinformation from the latest to those acquired in the past in apredetermined storage period (e.g., three months). Thus, when the healthinformation, the activity information, the environmental information, orthe emotion information is received at step S111, the server 3 updates,at step S112 and so as to include the received information, datacorresponding to the received information among the four types of data.

The storage period regarding the four types of data is the same for theserver 3. With this, the server 3 can perform an overall evaluation ofthe user on the basis of the QOL factor information in the storageperiod. In another embodiment, the storage period regarding the fourtypes of data may be any length, or may be different among the fourtypes of data.

The preference data is data indicating the preference informationacquired from the hand-held terminal 5 of the user. In the presentembodiment, the preference data indicates a genre, in which the user isinterested, assessed on the basis of the previously acquired preferenceinformation. Specifically, the preference data indicates the presence orabsence of interest of the user for each genre of preference describedabove.

Thus, when the preference information is received at step S111, theserver 3 updates, at step S112, the preference data so as to havecontents reflecting the received information. For example, whenpreference information indicating a certain genre has been acquired, theserver 3 updates the preference data so to have content indicatinginterest in the genre, if necessary. The preference data may begenerated on the basis of the previously acquired preferenceinformation, and the latest acquired preference information does notnecessarily have to be reflected immediately. For example, whenpreference information indicating the same genre has been acquired for apredetermined number of times (e.g., five times) or more in apredetermined period of time (e.g., one month), the server 3 may updatethe preference data so as to have content indicating interest in thegenre.

In another embodiment, similarly to the health data and the like, thepreference data may indicate the preference information from the latestto those acquired in the past in a predetermined storage period (e.g.,three months).

The server 3 may store, in addition to the user status data, informationused for calculating the information representing the user status data.More specifically, the hand-held terminal 5 may transmit the informationused for calculating the QOL factor information to the server 3 inaddition to transmitting the QOL factor information, and the server 3may store the received information. The information used for calculatingthe QOL factor information and stored in the server 3 may be presentedto the user on the hand-held terminal 5 through, for example, theprocess at step S119 described later (process of transmittinginformation from the server 3 to the hand-held terminal 5).

Details of the evaluation result data and privilege data shown in FIG.21 will be described later.

Returning to the description of FIG. 20, at step S113, the server 3judges whether or not the evaluation timing has arrived. In the presentembodiment, this judgment is performed depending on whether or not thepredetermined clock time determined in advance has arrived. When thejudgment result at step S113 is positive, the series of processes atstep S114 to S117 is executed. On the other hand, when the judgmentresult at step S113 is negative, the series of processes at steps S114to S117 is skipped, and the process at step S118 described later isexecuted.

At step S114, the server 3 performs an evaluation of the user on thebasis of the QOL factor information acquired from the hand-held terminal5. In the process at step S114, an evaluation is performed regardinghealth, environment, and emotion of the user. More specifically, theserver 3 calculates three factor indices, i.e., the health index, theenvironmental index, and the emotion index. In the following,calculation methods of these factor indices will be described.

(Calculation of Health Index)

First, a specific example of a calculation method of the health index(evaluation method regarding health) will be described. In the presentembodiment, the health index is calculated on the basis of healthinformation acquired when the user is awake (health informationcorresponding to the sleeping period) and activity information acquiredin the waking period of the user. More specifically, the server 3 firstcalculates the fatigue level in the waking period from the activityinformation (referred to as “second fatigue level” in distinction fromthe above described first fatigue level calculated from the biologicalinformation in the sleeping period), and calculates the health index onthe basis of the calculated second fatigue level and the healthinformation.

In the present embodiment, the server 3 stores, in advance, a tableshowing the association between a condition regarding the activityinformation and the change level of the second fatigue level, andcalculates the second fatigue level by using the table. FIG. 22 showsone example of a table used for calculating the second fatigue level. Asshown in FIG. 22, the table shows the association between a conditionregarding the activity information, and the change level of the secondfatigue level when the condition is satisfied. In the example shown inFIG. 22, the table associates a condition of the activity content being“work” and the activity level being “eight to ten hours” to a changelevel of the second fatigue level of “+10”. This means that whenactivity information showing the activity content is “work” and theactivity level is not less than eight hours but not less than ten hoursis acquired, the second fatigue level is increased by 10. The tableshown in FIG. 22 associates a condition of the activity content being“massage” but without the activity level being designated, to a changelevel of “−20”. This means that when the activity content is “massage”,the second fatigue level is lowered by 20. In the table described above,a condition that increases the second fatigue level and a condition thatlowers the second fatigue level may both be included, or only either oneof the two may be included.

The server 3 calculates the change level of the second fatigue levelcorresponding to the acquired activity information by referring to thetable. For example, when activity information in which the activitycontent is “movement (walking)” and the activity level is “ten minutes”is acquired, the server 3 calculates “+10” as the change level of thesecond fatigue level corresponding to this activity information. In thepresent embodiment, the server 3 calculates the change levelcorresponding to each set of activity information acquired in apredetermined period (in the present embodiment, today).

Furthermore, the server 3 calculates a final second fatigue level on thebasis of a predetermined basic fatigue level (e.g., “0”) and the changelevel of the calculated second fatigue level. Specifically, the finalsecond fatigue level is calculated by making addition or subtraction inaccordance with the change level of the second fatigue level withrespect to the basic fatigue level. For example, in a case where thebasic fatigue level is “0”, when the change level is calculated as“+10”, “+15”, and “−10”, the final second fatigue level is calculated as“+15”. It should be noted that the server 3 may use the value the firstfatigue level (fatigue index) indicated by the health information as thebasic fatigue level.

When the second fatigue level is calculated, the server 3 calculates thehealth index on the basis of the second fatigue level and the healthinformation. The health index is calculated in reflection of the valuesof the respective fatigue levels (first fatigue level and second fatiguelevel). For example, the health index is calculated such that the healthindex becomes larger when each of the fatigue levels (first fatiguelevel and second fatigue level) is smaller. In addition, the healthindex is calculated in reflection of the values of sleep indicesindicated by the health information. For example, the health index iscalculated such that the health index becomes higher when the sleepindices indicate that the quality of sleep is good. For example, theserver 3 may calculate, as the overall fatigue level, a value obtainedas a sum of the fatigue levels each assigned with a predeterminedweight, and calculate the health index by making addition or subtractionin accordance with the sleep indices with respect to the calculatedoverall fatigue level. In the present embodiment, the health index iscalculated as an integer from 0 to 100, and a larger numerical value ofthe health index represents being healthier.

In another embodiment, the health index may be calculated on the basisof other QOL factor information. For example, the health index may becalculated on the basis of the environmental information in addition tothe activity information. More specifically, even when the activityinformation is the same content, the server 3 may calculate the healthindex with different values depending on the environmental information.For example, when the environmental information indicates that theenvironment is relatively not good for the user (e.g., when ambienttemperature and humidity are too high), the server 3 may calculated thesecond fatigue level to be high when compared to a case in which theenvironmental information indicates that the environment is relativelygood for the user (e.g., when ambient temperature and humidity are atsuitable levels).

In another embodiment, the health index may be calculated on the basisof, in addition to the activity information, the preference information.For example, when activity information indicates physical exercise, theserver 3 may calculate the value of the second fatigue level to berelatively low when the preference information indicates that the userhas interest in physical exercise, or calculate the value of the secondfatigue level to be relatively high when the preference informationindicates that the user has no interest in physical exercise. In thepresent embodiment, although the preference information is not used foran evaluation regarding the user (calculation of QOL index), thepreference information may be used for an evaluation regarding the user(calculation of QOL index) as described above.

(Calculation of Environmental Index)

Next, a specific example of a calculation method of the environmentalindex (evaluation method regarding environment) will be described. Inthe present embodiment, the environmental index is calculated on thebasis of environmental information acquired in the waking period of theuser. In the present embodiment, the server 3 stores, in advance, atable showing the association between a condition regarding theenvironmental information and the change level of the environmentalindex, and calculates the environmental index by using the table.

FIG. 23 shows one example of the table used for calculating theenvironmental index. As shown in FIG. 23, the table shows theassociation between a condition regarding the environmental informationand the change level of the environmental index when the condition issatisfied. The condition regarding the environmental informationincludes an item (condition) regarding activity content and an item(condition) regarding the environmental value. In the example shown inFIG. 23, the table associates a condition showing that the activitycontent is “movement (walking)” and the environmental value is “ambienttemperature: not lower than 30° C. but lower than 33° C., and humidity:not lower than 60% but lower than 70%”, with respect to the change levelof the environmental index of “−15”. This means that when environmentalinformation satisfying the condition of the activity content being“movement (walking)” and the environmental value being “ambienttemperature: not lower than 30° C. but lower than 33° C., and humidity:not lower than 60% but lower than 70%” is acquired, the environmentalindex is to be decreased by 15. In addition, the table associates acondition showing that the activity content is “work” and theenvironmental value is “ambient temperature: lower than 28° C., andhumidity: lower than 60%”, with respect to the change level of theenvironmental index of “+10”. This means that when environmentalinformation satisfying the condition of the activity content being“work” and the environmental value being “ambient temperature: lowerthan 28° C., and humidity: lower than 60%” is acquired, theenvironmental index is to be increased by 10. As described above, in thetable, a condition that increases the environmental index and acondition that lowers the environmental index may both be included, oronly either one of the two may be included.

The server 3 calculates the change level of the environmental indexcorresponding to the acquired environmental information by referring tothe table. For example, when environmental information in which theactivity content is “movement (walking)” and the environmental value is“ambient temperature: 31° C. and humidity 65%” is acquired, the server 3calculates “−15” as the change level (of the environmental index)corresponding to the environmental information. In the presentembodiment, the server 3 calculates corresponding change levels for eachset of environmental information acquired within a predetermined periodof time (in the present embodiment, within today).

Furthermore, the server 3 calculates the environmental index on thebasis of a predetermined basic value (e.g., “50”) and the change levelof the calculated environmental index. Specifically, a finalenvironmental index is calculated by making addition or subtraction inaccordance with the change level of the environmental index with respectto the basic value. For example, when the basic value is “50” and changelevels are calculated as “−10”, “−15”, and “+5”, a final environmentalindex is calculated as “30”. The basic value described above may be apredetermined fixed value, or may be calculated on the basis of theenvironment sensor information sensed during the waking period. When theserver 3 can acquire the weather of the location of the user, the basicvalue may be set variably in accordance with the weather. Morespecifically, the server 3 may calculate the environmental index thattakes into consideration the weather of the location of the user. In thepresent embodiment, environmental index is calculated as an integer from0 to 100, and a larger numerical value of the environmental indexrepresents the environment around the user being more comfortable.

In another embodiment, the environmental index may be calculated on thebasis of the season and/or area (positional information). For example,the server 3 may specify the area and season on the basis of informationregarding the current date and time and positional information of thehand-held terminal 5. The server 3 may prepare multiple types of tablesfor each area and season as the table for calculating the change levelof the environmental index, and calculate the change level describedabove by using a table corresponding to the specified area and season.

(Calculation of Emotion Index)

Next, a specific example of a calculation method of the emotion index(evaluation method regarding emotion) will be described. In the presentembodiment, the emotion index is calculated on the basis of emotioninformation acquired in the waking period of the user. As describedabove, in the present embodiment, the emotion information includesrespective indices indicating the degree of three of five types ofemotions (anger, joy, sadness, hate, and pleasure). The server 3calculates the emotion index on the basis of each of the indices.Specifically, the server 3 calculates the emotion index such that largerindices of negative emotions (in the present embodiment, anger sadness,and hate) result in a smaller value (of the emotion index). In addition,the server 3 calculates the emotion index such that larger indices ofpositive emotions (in the present embodiment, joy and pleasure) resultin a larger value (of the emotion index). For example, the server 3 maycalculate the emotion index by making, with respect to a predeterminedbasic value (e.g., “50”), a subtraction by an amount in accordance withthe index indicating the negative emotions, and an addition by an amountin accordance with the index indicating the positive emotions. The basicvalue may be a predetermined fixed valued, or may be set variably inaccordance with the user's schedule information (e.g., informationrepresenting whether being holiday or not, busy or not, etc.), activityinformation, and/or weather. The basic value may be set, for example,relatively high for a holiday and relatively low for a work day.Furthermore, for example, the basic value may be set high on a day inwhich the user is performing a recreational (leisure) activity, and maybe set low on a day in which the user is performing a work activity.Still further, for example, the basic value may be set high on a fineweather day and low on a bad weather day. In the manner described above,in the present embodiment, the emotion index is calculated to be largewhen the emotion of the user is more positive. In the presentembodiment, the emotion index is calculated as an integer from 0 to 100.

At step S115, the server 3 calculates a comprehensive index (QOL index)on the basis of each of the factor indices calculated at step S114. TheQOL index may be calculated with any method that calculates the QOLindex so as to have a level reflecting each of the factor indices. Forexample, in the present embodiment, the server 3 calculates, as the QOLindex, a value obtained by adding up the factor indices each assignedwith a predetermined weight. In the present embodiment, the QOL index iscalculated as an integer from 0 to 100.

When each of the indices is calculated at steps S114 and S115, theserver 3 stores (accumulates) evaluation result data indicating each ofthe calculated indices. As shown in FIG. 21, the evaluation result dataincludes health index data indicating the health index, environmentalindex data indicating the environmental index, emotion index dataindicating the emotion index, and QOL index data indicating the QOLindex. In the present embodiment, these four types of index dataindicate, for example, respective indices from the latest to thosecalculated in the past in a predetermined storage period (e.g., threemonths). Thus, when each of the indices is calculated at steps S114 andS115, the server 3 updates each set of the index data so as to have acontent including each of the newly calculated indices.

At step S116, the server 3 specifies the content of the network servicein accordance with the evaluation result. More specifically, the server3 specifies the content of the network service to be provided to theuser on the basis of the evaluation result obtained at step S114 and/orS115. In the present embodiment, a service of presenting the evaluationresult is provided regardless of the content of the evaluation result.Regarding a service other than presenting the evaluation result(providing advice information, recommendation information, and content,and service regarding giving a privilege); whether to provide theservice or not and what content is to be provided are specified inaccordance with the evaluation result.

In the present embodiment, the server 3 stores, in advance, a tableshowing the association between a condition regarding the evaluationresult (each index) and service content to be provided, and specifiesthe service content by using the table. FIG. 24 shows one example of atable used for determining the network service in accordance with theevaluation result. As shown in FIG. 24, the table shows the associationbetween a condition regarding each of the indices (QOL factor indicesand QOL index), and a service content to be provided when the conditionis satisfied. It should be noted that, in FIG. 24, “-” shows that acondition has not been set.

In the example shown in FIG. 24, the table includes a set in which acondition of the QOL index shown to be not larger than 30 is associatedwith advice information of advice A and recommendation informationregarding commodity A ((a) shown in FIG. 24). In this manner, thecondition regarding the evaluation result may include a conditionregarding the QOL index. Thus, the server 3 may specify the content ofthe network service on the basis of the QOL index.

Furthermore, in the example shown in FIG. 24, the table includes a setin which a condition of the health index being not larger than 30 andthe emotion index being not larger than 30 is associated with adviceinformation of advice B and recommendation information regardingcommodity B ((b) shown in FIG. 24). In this manner, the conditionregarding the evaluation result may include a condition regarding acombination of each index (factor index and/or comprehensive index).Thus, the server 3 may specify the content of the network service on thebasis of two or more indices.

Furthermore, in the example shown in FIG. 24, the table includes a setin which a condition of the health index being not larger than 30, theenvironmental index being not larger than 30, and the preferenceinformation showing that the user has interest in physical exercise isassociated with advice information of advice C ((d) shown in FIG. 24).In this manner, the condition regarding the evaluation result mayinclude a condition regarding the preference information. Thus, inaddition to the evaluation result (each index), the server 3 may specifythe content of the network service on the basis of the preferenceinformation.

Furthermore, in the example shown in FIG. 24, the table includes a setin which a condition showing the health index to be not smaller than 70is associated with addition of points ((f) shown in FIG. 24). In themanner described above, the server 3 gives a privilege to the user onthe basis of the evaluation result regarding health (more specifically,under a condition that the evaluation regarding health is better than apredetermined standard).

The server 3 specifies the content of the network service to be providedby referring to the table and on the basis of the calculated evaluationresult. More specifically, when the calculated evaluation resultsatisfies a condition included in the table, the server 3 provides anetwork service associated with the condition. Although notdiagrammatically represented, the server 3 accumulates and stores, inthe storage section, data indicating a specified service for each user.Thus, the server 3 stores information indicating the service content tobe provided with respect to the user.

The server 3 accumulates information regarding the user (the user statusdata and the evaluation result data) in the storage section, morespecifically, stores past information of the user. Thus, in anotherembodiment, the server 3 may determine the content of the service datato be provided to a certain user, on the basis of past information (theuser status data and/or the evaluation result data) accumulated in thestorage section regarding the user. For example, the server 3 maydetermine the service content on the basis of evaluation result of thepast one week (e.g., an average value of indices over the past oneweek), or may determine the service content on the basis of thedifference between the present evaluation result and the past evaluationresult (e.g., the level of change of an index of the current week fromthat in the last week).

In addition, the server 3 accumulates information regarding multipleusers (the user status data and the evaluation result data) in thestorage section for each of the users. Thus, the server 3 may determinethe content of the service data to be provided to a certain user on thebasis of information (the user status data and/or the evaluation resultdata) accumulated in the storage section regarding another user who isdifferent from the certain user. For example, the server 3 may calculatean average value of an index for predetermined multiple users, anddetermine the service content on the basis of a relationship between theaverage value and an index of a certain user. It should be noted thatthe user (user for which an average value is to be calculated) whoseinformation is used for determining the service content is not limitedto all users subscribing to the network service, and may be certainusers satisfying a predetermined condition (e.g., users in the same agegroup, users living in the same area, users of which a calculated indexis in the same range).

Furthermore, for example, the server 3 may determine the content of theservice data to be provided to a certain user on the basis of pastinformation (the user status data and the evaluation result data)regarding another user different from the certain user. For example, theserver 3 calculates the change of a predetermined index (e.g., level ofdeterioration of health index, etc.) in a predetermined period of timefrom the present to the past for predetermined multiple users. Then, theserver 3 specifies another user having the same trend of the calculatedchange as the certain user to which the service is to be provided (e.g.,the health index being deteriorated by equal to or larger than apredetermined ratio in a single month, etc.). Furthermore, the server 3determines the service content to be provided to the certain user on thebasis of the service content provided in the past to the specified otheruser. In this manner, the server 3 may determine the service content tobe provided to the user, by using past information of another user andreferring to the service content provided to the other user whose changein index has the same trend as the user.

At step S116, the server 3 may specify the service content on the basisof whether or not the evaluation result has been improved from apredetermined standard. For example, the server 3 may judge whether ornot the evaluation result (various types of indices calculated at stepsS114 and S115) has been improved from a predetermined standard, andprovide a predetermined network service (e.g., give a privilege) to theuser under a condition that the evaluation result is judged to beimproved from the predetermined standard. The specific content of thepredetermined standard may be any content, and, for example, apredetermined value (fixed value) may be used, or a value based on pastevaluation result (indices) may be used. For example, the server 3 mayuse, as a standard, an average value of the last one week (or a valueobtained by adding a predetermined value to the average value). Thus,the server 3 may give a privilege to the user under a condition that anindex calculated recently is higher than an average value of the indexin the last week.

In the present embodiment, the server 3 repeatedly provides (transmits)the service data (advice information, etc.) to the hand-held terminal 5.Thus, additional service is provided on the basis of the evaluationresult of the user after a certain service is provided. With this, sinceservice data such as advice information is repeatedly (continuously)provided to the user, the user can be effectively encouraged to improveQOL.

The server 3 may compare the evaluation result between before and afterproviding certain service data to the hand-held terminal 5, and specifycontent of a service to be provided newly (additional service) on thebasis of the comparison result. More specifically, as a result of thecomparison, the server 3 may provide services with different contents ina case where the evaluation result is judged to be improved and in acase where evaluation result is judged not to be improved. For example,the server 3 may calculate the difference in the evaluation result(index) between before and after providing certain advice information,and specify, on the basis of the calculated difference, the content ofadvice information to be provided after providing the certain adviceinformation.

In another embodiment, from the evaluation result, the server 3 mayupdate the condition (table shown in FIG. 24 in the above describedembodiment) for determining the service content to be provided on thebasis of an evaluation result of one or more users. For example, whenthe advice information is presented to a user, the server 3 may comparethe evaluation result of the user between before and after presentingthe advice information, and update the condition regarding the adviceinformation on the basis of a comparison result. For example, as aresult of the comparison, when the server 3 assesses that the evaluationresult (e.g., QOL index) of the user has improved between before andafter presenting the advice information, the server 3 may update thecondition such that the advice information is more likely to bepresented. The judgment of whether or not to update the condition may beperformed on the basis of evaluation results regarding multiple users.For example, when a predetermined number or more (predetermined ratio orhigher) users had improved evaluation results among multiple userspresented with a certain advice information, the server 3 may update thecondition such that the advice information is more likely to bepresented. With this, since the condition for providing the adviceinformation can be altered depending on the actual effect, moreappropriate advice information can be presented to the user.

Similarly to the case with the advice information, the server 3 canupdate the condition on the basis of an evaluation result of one or moreusers for services other than the advice information (providerecommendation information and/or content). Regarding the recommendationinformation, for example, the server 3 may compare the evaluation resultbetween before and after the user purchases a commodity as a result ofthe recommendation information, and update the condition regarding therecommendation information on the basis of the result of the comparison.Regarding providing of content, the server 3 may compare the evaluationresult between before and after the content is provided, and update thecondition regarding providing of the content on the basis of the resultof the comparison.

At step S117, the server 3 transmits, to the hand-held terminal 5,service data related to the network service to be provided. Morespecifically, the server 3 transmits data representing an evaluationresult based on steps S114 and S115 as the service data to the hand-heldterminal 5. Furthermore, the server 3 transmits, to the hand-heldterminal 5, service data regarding the service content specified at stepS116. For example, when providing a service of presenting the adviceinformation, the server 3 transmits data indicating the adviceinformation to the hand-held terminal 5 as the service data. Forexample, when providing a service of presenting the recommendationinformation, the server 3 transmits data indicating the recommendationinformation to the hand-held terminal 5 as the service data. Forexample, when providing a service of providing content, the server 3transmits data of the content to the hand-held terminal 5 as the servicedata. For example, when providing a service of giving a privilege, theserver 3 transmits data indicating a notification of giving theprivilege to the hand-held terminal 5 as the service data.

The server 3 stores, in the storage section for each user, data ofprivilege information (privilege data) indicating a privilege to begiven to a user (see FIG. 21). At step S117, when providing the serviceof giving a privilege, the server 3 updates the privilege data stored inthe storage section so as to have a content indicating the privilege tobe given. The privilege data is referred to by an information processingapparatus (shop server and billing server described later) that executesa process related to a privilege (process of actually giving aprivilege, such as, for example, billing process and purchasing process)when actually giving the privilege. The user utilizes the privilege, andbilling or calculation of charge is performed in consideration of theprivilege data.

For example, when the privilege relates to purchasing of a commodity orthe like, the shop server refers to the privilege data when the userpurchases a commodity or the like, next. Here, the shop server is aserver that manages purchase, billing, and providing of a commodity orthe like, and may be the same server as, or a different server from, theserver 3. For example, when the user purchases a commodity or the like,shop server calculates, as a billing amount, an amount obtained afterdiscounting an amount of money in accordance with the privilege data offthe amount of money of the purchased commodity or the like. Theprivilege data may indicate points that can be used when purchasing thecommodity or the like, or may indicate the amount of discount.

For example, when the privilege relates to usage charge of the hand-heldterminal 5, the billing server refers to the privilege data for thebilling of the usage charge. The billing server is a server thatperforms a billing process by calculating the usage charge of thehand-held terminal 5, and may be the same server as, or a differentserver from, the server 3. For example, when calculating the usagecharge of the hand-held terminal 5, the billing server calculates, asthe usage charge, an amount obtained after discounting an amount ofmoney in accordance with the privilege data off a predetermined usagecharge. The privilege data may indicate points that can be used for theusage charge, or may indicate the amount of discount.

In the present embodiment, the server 3 calculates a combined usagecharge for the use of communication function (a function of performingcommunication through a mobile phone communication network, and alsoincludes a telephone call function by the communication) on thehand-held terminal 5, and the use of the network service on thehand-held terminal 5. As a result, the user can conveniently paytelephone call charges and the usage charge of the network servicetogether. Specifically, the server 3 stores, in the storage section,usage charge data indicating the usage charge for each user. The billingserver refers to the usage charge data stored in the storage section ofthe server 3 and executes the billing process. The calculation methodfor the usage charge may be any method, and the server 3 may apply afixed amount of usage charge for a unit period, or may calculate theusage charge in accordance with a communication volume and/or the usageamount of the network service.

In the present embodiment, at step S117, the server 3 transmits theservice data to the hand-held terminal 5 of the user who is the subjectof evaluation, and to a terminal of the notification-subject user setfor the user. More specifically, the server 3 transmits the service datato the terminal of the notification-subject user indicated by thenotification user data containing the user data. Although details willbe described later, the service data transmitted to the hand-heldterminal 5 of the user who is the subject of evaluation and the servicedata transmitted to the terminal of the notification-subject user set bythe user may be the same or different. Thus, the server 3 may providenetwork services with different contents to the user who is the subjectof evaluation and to the notification-subject user.

At step S118, the server 3 judges whether or not there has been arequest from the hand-held terminal 5 of the user. More specifically, atstep S102, the server 3 judges whether or not a request transmitted fromthe hand-held terminal 5 has been received. When the judgment result atstep S118 is positive, the process at step S119 is executed. On theother hand, when the judgment result at step S118 is negative, theprocess at step S111 is executed once again.

At step S119, the server 3 executes a process in accordance with therequest received at step S118. For example, when the request is arequest to login to a website, the server 3 transmits, to the hand-heldterminal 5, information of an image (web page) for login. Furthermore,for example, when the request is a request to acquire a web page, theserver 3 transmits information of the web page to the hand-held terminal5. On the hand-held terminal 5, the processes at steps S103 and S104 areexecuted in accordance with the process on the server 3 at step S119.

Next after step S119, the server 3 executes the process at step S111once again. The server 3 repeatedly executes the series of processes atsteps S111 to S119. It should be noted that the server 3 executes theseries of processes at steps S111 to S119 with respect to each hand-heldterminal carried by multiple users who receive the network service.

6. Specific Example of Information to be Presented

Next, a specific example of information presented to the user throughthe above described processing action of the information processingsystem 1 will be described. As described above, the server 3 performs anevaluation of the user at a predetermined timing in a single day, andtransmits, to the hand-held terminal 5, the service data associated withthe network service in accordance with the evaluation result. On thehand-held terminal 5, information based on the received service data ispresented to the user.

FIG. 25 shows one example of the information presented on the hand-heldterminal 5. In the present embodiment, at step S36 (step S117) describedabove, when the service data is transmitted from the server 3 to thehand-held terminal 5, the hand-held terminal 5 displays, for example, animage shown in FIG. 25 on the display 17 on the basis of the servicedata received at step S37 (step S104) described above.

As shown in FIG. 25, a QOL score image 50 and a face image 51 aredisplayed on the display 17. The QOL score image 50 represents anevaluation result of an overall evaluation, and specifically representsa numerical value (score) of the QOL index. On display, the QOL scoreimage 50 is displayed as “(Today's) QOL index number” in order to beeasily understood by the user.

The face image 51 is displayed such that the display style thereof(specifically, facial expression) is changed depending on the QOL index.For example, when the QOL index is at a moderate level, a face imagerepresenting an ordinary facial expression is displayed; when the QOLindex is relatively high (good numerical value), a face imagerepresenting a smiley facial expression is displayed; and when the QOLindex is relatively low (bad numerical value), a face image representinga fatigued facial expression is displayed as shown in FIG. 25. Bydisplaying the face image 51 in such manner as the evaluation result,the user can recognize the level of QOL as the evaluation result,intuitively.

As described above, when the numerical value of the QOL index iscalculated as the evaluation result, the numerical value itself may bepresented to the user or information representing the magnitude of thenumerical value may be presented to the user. By presenting suchinformation, the level of QOL which is the evaluation result can bepresented to the user in an easily understandable manner. Other than thescore and image described above, the presented information may representthe fatigue level. For example, in another embodiment, informationrepresenting the QOL index in five grades of A to E may be presented.

As shown in FIG. 25, various types of score images 52 to 54 aredisplayed on the display 17. These score images 52 to 54 respectivelyrepresent numerical values (scores) of the factor indices describedabove. The health score image 52 represents a numerical value of thehealth index which is a result of the evaluation regarding health of theuser, and, on display, is displayed as “vigor level” in order to beeasily understood by the user. The environmental score image 53represents a numerical value of the environmental index which is theevaluation result regarding environment of the user, and, on display, isdisplayed as “comfort level” in order to be easily understood by theuser. The emotion score image 54 represents a numerical value of theemotion index which is the evaluation result regarding the environmentof the user, and, on display, is displayed as “emotion level” in orderto be easily understood by the user.

Detail buttons 55 to 57 associated with the respective score images 52to 54 are displayed on the display 17. The detail button 55 isassociated with the health score image 52, and is a button fordisplaying further detailed information of the health index. The detailbutton 56 is associated with the environmental score image 53, and is abutton for displaying further detailed information of the environmentalindex. The detail button 57 is associated with the emotion score image54, and is a button for displaying further detailed information of theemotion index. In response to the user making an input (e.g., touching)with respect to each of the detail buttons 55 to 57, the hand-heldterminal 5 displays, instead of the screen shown in FIG. 25, detailedinformation (described later with reference to FIG. 26) for the indexassociated to the detail button on the display 17.

In addition, an advice image 58 representing the advice information isdisplayed on the display 17. The advice image 58 represents the adviceinformation corresponding to the evaluation result (each of the scoreimages 50, and 52 to 54) that is presented (displayed) simultaneously.The advice information may be, among multiple evaluation resultspresented simultaneously, information representing an advicecorresponding to the overall evaluation, or may be an advicecorresponding to some (or one) of the evaluations. For example, theadvice image 58 shown in FIG. 25 relates to the health index shown bythe health score image 52 being relatively low and represents an advicefor improving the health index.

In addition, a recommendation information button 59 is displayed on thedisplay 17. The recommendation information button 59 is a button fordisplaying the recommendation information corresponding to theevaluation result (each of the score images 50, and 52 to 54) that ispresented (displayed) simultaneously. More specifically, in response tothe user making an input with respect to the recommendation informationbutton 59, the hand-held terminal 5 displays the recommendationinformation on the display 17 instead of the screen shown in FIG. 25. Inanother embodiment, the recommendation information corresponding to theevaluation result may be displayed in addition to the evaluation result,instead of the recommendation information button 59.

A specific example of the image displayed on the hand-held terminal 5when an input is made with respect to the detail button will bedescribed next. FIG. 26 shows one example of the image displayed on thedisplay 17 when an input is made with respect to the detail button 55shown in FIG. 25.

In FIG. 26, a health score image 61 is displayed on the display 17.Similarly to the health score image 52 shown in FIG. 25, the healthscore image 61 represents a numerical value of the health index.However, the health score image 61 shown in FIG. 26 contains informationregarding the date on which the evaluation has been performed.

In addition, a sleep-state score image 62 and an awake-state score image63 are displayed on the display 17. The sleep-state score image 62represents a health index (referred to as “sleep-state health index”)calculated based on information (biological information) sensed duringthe sleeping period. In the present embodiment, the sleep-state healthindex is calculated by the server 3 on the basis of the healthinformation described above. More specifically, the sleep-state healthindex is calculated on the basis of a sleep index and a fatigue index(first fatigue level) included in the health information, and iscalculated as, for example, a total value of the fatigue index and thesleep index each assigned with a predetermined weight.

The awake-state score image 63 represents a health index (referred to as“awake-state health index”) calculated based on information (positionalinformation and environmental information) sensed during the wakingperiod. In the present embodiment, the awake-state health index iscalculated by the server 3 on the basis of the activity informationdescribed above. For example, as the awake-state health index, thenumerical value of the second fatigue level calculated on the basis ofthe activity information may be used.

On display, the score images 61 to 63 are displayed as “vigor level” inorder to be easily understood by the user, similarly to the health scoreimage 52.

The sleep-state score image 62 and the awake-state score image 63 eachincludes a detail button (see FIG. 26). When an input is made withrespect to any of these detail buttons, the hand-held terminal 5displays further detailed information regarding an index represented byeach of the score images. For example, the hand-held terminal 5 maydisplay information used for calculating an index (e.g., for thesleep-state health index, various types of sleep indices or, for theawake-state health index, the activity information used for calculatingthe awake-state health index).

In addition, a graph display button 64 is displayed on the display 17.The graph display button 64 is a button for displaying a graph relatedto the health index. More specifically, in response to the user makingan input with respect to the graph display button 64, the hand-heldterminal 5 displays the graph on the display 17 instead of the screenshown in FIG. 26. For example, the graph may be a graph showing thechange in the health index, the sleep-state health index, and theawake-state health index in a predetermined period of time (e.g., in thelast week).

In addition, a recommendation information button 65 is displayed on thedisplay 17. The recommendation information button 65 is a button fordisplaying the recommendation information, similarly to therecommendation information button 59 shown in FIG. 25. However, therecommendation information button 65 shown in FIG. 26 is a button fordisplaying the recommendation information regarding the health index(may be sleep-state health index or awake-state health index). Morespecifically, in response to the user making an input with respect tothe recommendation information button 65, the hand-held terminal 5displays the recommendation information regarding a commodity or thelike for improving the health index on the display 17, instead of thescreen shown in FIG. 26. When the recommendation information to bepresented depending on the evaluation result includes a content relatedto the health index (related to a commodity or the like for improvingthe health index), the recommendation information button 65 may bedisplayed; and when said recommendation information includes a contentrelated to an index other than the health index, the recommendationinformation button 65 may be not displayed.

A specific example of the image displayed on the hand-held terminal 5when the privilege is to be given to the user as the network servicewill be described next. FIG. 27 shows one example of the image displayedon the display 17 when the privilege is to be given to the user.

As shown in FIG. 27, also when the privilege is to be given, the images51 to 59 are each displayed on the display 17 similarly to when otherservices are to be provided. However, instead of the advice information,in FIG. 27, the advice image 58 represents a message notifying that theprivilege has been given. In such manner, when the privilege is to begiven, a notification showing that the privilege has been given may bepresented to the user.

A specific example of the image displayed on the terminal of thenotification-subject user will be described next. FIG. 28 shows oneexample of the image displayed on the terminal of thenotification-subject user. The image shown in FIG. 28 is displayed onthe terminal of the notification-subject user when the image shown inFIG. 25 is displayed on the hand-held terminal 5 of the user who is thesubject of evaluation.

As shown in FIG. 28, also on the terminal of the notification-subjectuser, the images 51 to 59 are each displayed similarly on the hand-heldterminal 5 of the user who is the subject of evaluation. The image onthe terminal of the notification-subject user is different from theimage shown in FIG. 25 in terms of an image 66 showing the name of thenotification-subject user being displayed. On the terminal of thenotification-subject user, the content of the advice information shownby the advice image 58 are different from the content displayed on thehand-held terminal 5 of the user who is the subject of evaluation. Insuch manner, by changing the advice for the user who is the subject ofevaluation and the advice of the notification-subject user who is notthe said user, a more appropriate advice can be presented.

As described above, the server 3 may change the advice information(similarly to the recommendation information) depending on the user towhom the advice information is to be presented. For example, a case willbe discussed in which the user of the hand-held terminal 5 had a lowemotion index since being scolded by his/her superior at work. In thiscase, the server 3 may present the user with advice information of “takea deep breath” or present recommendation information introducing stressalleviation goods. On the other hand, the server 3 may present thenotification-subject user with advice information such as “you shouldcheer up his/her feelings” or “you should cook his/her favorite dish”.

Furthermore, the server 3 may present notification-subject users withdifferent information (advice information and/or recommendationinformation) in accordance with the respective notification-subjectuser's relationships (parent, child, wife, sibling, family, or friend,etc.) with the user who is the subject of evaluation. For example, inthe case of the example described above, when the notification-subjectuser is a family member, advice information such as “you should cookhis/her favorite dish when he/she returns home” may be presented, and,when the notification-subject user is a friend, advice information suchas “you should telephone him/her and listen to his/her story” may bepresented. By having the server 3 store notification user data includinginformation indicating the relationship between the user who is thesubject of evaluation and notification-subject users, the server 3 canspecify the relationship.

In FIGS. 25 to 28, when a button (e.g., buttons 55 to 57 or the likeshown in FIG. 25) for displaying another image is contained in the imagedisplayed on the hand-held terminal 5, information (the other image)displayed when an input is made with respect to the button may beacquired from the server 3 at any timing. More specifically, theinformation may be transmitted from the server 3 to the hand-heldterminal 5 in advance as the service data, or may be transmitted fromthe server 3 to the hand-held terminal 5 in response to a request madeto the server 3 by the hand-held terminal 5 when an input is made withrespect to the button. Thus, the information may be transmitted to thehand-held terminal 5 by the process at step S117 or may be transmittedto the hand-held terminal 5 by the process at step S119.

7. Functions and Effects of Present Embodiment

The information processing system 1 of the embodiment described aboveexerts the advantageous effects as described next.

Since biological information (pulse, respiration, etc.) is acquired(measured) without any contact with the user, the user does not have totouch a sensor.

Since various types of information for calculating the QOL factorinformation are acquired by the hand-held terminal 5 and the base device6, the user does not have to wear a special device on the body.

Since processes for acquiring (measuring) the various types ofinformation are initiated without any operation by the user, the userdoes not have to perform an operation to cause the terminal system 2 toacquire the various types of information.

Since the various types of information are automatically calculated andan evaluation based on the various types of information is automaticallyperformed, the user does not have to wait until the information isacquired (measured) and the evaluation result is calculated, and doesnot have to make any special preparation every time for acquiring theinformation.

As a result, with the embodiment described above, the user can receive aservice based on the evaluation of QOL by simply spending his/her lifeas usual without performing any troublesome operation or tasks. Thus,the service provided in the present embodiment can be continuously usedeasily by the user. Since providing a service based on the evaluation ofQOL is considered to be effective when being continuously performed withrespect to the user, the information processing system 1 can provide amore effective service by providing the service with a method with whichthe user can easily continue, as in the present embodiment.

(Functions and Effects Related to Configuration of InformationProcessing System)

In the embodiment described above, the information processing system 1includes the stationary type device (the base device 6), the hand-heldtype device (the hand-held terminal 5) which is a multifunctionalterminal connectable to the base device, and the server 3 capable ofcommunicating with the hand-held terminal 5 through a wide area network(the Internet and/or a mobile communication network). The base device 6is installed around the user in bed, and senses biological informationof the user in sleep (step S12). The hand-held terminal 5 transmits, tothe server 3 as transmission information, information (healthinformation, and may be biological information itself) calculated fromthe biological information sensed by the base device 6 (steps S4, S19).The server 3 transmits, to the hand-held terminal 5, service dataassociated with a network service for improving an evaluation result(health index) of an evaluation (calculation of health index) of sleepof the user performed on the basis of the biological information (stepsS9, S117). The hand-held terminal 5 presents the user with an imageand/or a sound associated with the service data received from the server3 regarding a predetermined network service at least in a state of notbeing connected with the base device 6 (steps S37, S104).

With the configuration described above, the information processingsystem 1 can evaluate health of the user on the basis of the biologicalinformation of the user in sleep, and provide the user with a networkservice in accordance with the evaluation result. Since the base device6 does not require means for communicating with the server 3 becauseinformation is transmitted to the server 3 by the hand-held terminal 5,the configuration of the base device 6 can be simplified. Since thehand-held terminal 5 can present an image and/or a sound associated withthe service data in a state of not being connected with the base device6, the user can easily receive the provided service.

In the embodiment described above, the stationary base device 6 sensesuser information (biological information) for evaluating QOL of the user(step S12). The hand-held terminal 5 transmits, to the server 3 astransmission information, information (health information, or may be theuser information itself) calculated from the user information sensed bythe base device 6 (steps S4, S19). The server 3 transmits, to thehand-held terminal 5, service data for providing the user with a networkservice in accordance with an evaluation result (evaluation resultregarding QOL) (QOL index) calculated from the transmission informationtransmitted from the hand-held terminal 5 (or an evaluation resultindicated by the transmission information) (steps S9, S117). With this,the information processing system 1 can evaluate QOL of the user on thebasis of the user information, and provide the user with a networkservice in accordance with the evaluation result. In the descriptionabove, the hand-held terminal 5 may present the user with information(information of evaluation result, advice information, recommendationinformation, information of content included in the service data, andinformation regarding a privilege, etc.) based on the service datareceived from the server 3.

In the embodiment described above, the information processing system 1calculates, as the evaluation result regarding QOL, the QOL index on thebasis of three types of information regarding health, environment, andemotion of the user. The evaluation regarding QOL may be performed onthe basis of at least one of the three types of information. Forexample, in another embodiment, an evaluation result (health indexdescribed above) regarding health of the user may be calculated as theQOL index. Furthermore, the evaluation regarding QOL may be performed onthe basis of other information different from the three types ofinformation. For example, in another embodiment, the QOL index may becalculated on the basis of information (preference information describedabove) regarding preference of the user.

In the embodiment described above, the stationary base device 6 isinstalled around the user in bed, and senses user information(biological information) of the user in sleep for evaluating QOL of theuser (step S12). The hand-held terminal 5 transmits, to the server 3 astransmission information, information calculated from user informationsensed by the base device 6 (steps S4, S19). The server 3 accumulates,in a predetermined storage section as accumulation information,information (health information, or may be transmission informationitself) calculated from the transmission information transmitted fromthe hand-held terminal 5 (step S112).

With this, the information processing system 1 can calculate theinformation regarding QOL of the user on the basis of the informationsensed from the user in sleep. Furthermore, the QOL information isuseful information since various types of services can be provided tothe user by analyzing or evaluating the information, and becomes furtheruseful information if multiple sets of the information (may be multiplesets of the information for a single user or may be information formultiple users) of QOL exist. In the embodiment described above, sincethe calculated QOL information is accumulated in the server 3, a largenumber of sets of the QOL information, which is useful information, canbe easily managed without placing a large burden on the storage capacityof the hand-held terminal 5.

The base device 6 initiates sensing of the user information (biologicalinformation) in response to becoming communicable with the hand-heldterminal 5 (see FIG. 6). With this, since the user does not have toperform an operation for causing the base device 6 to initiate sensing,the time and effort required for the operation by the user can beomitted. In addition, the base device 6 can transmit the informationbased on the sensed user information to the hand-held terminal 5 withoutstoring information in itself for a long period of time. The embodimentrelates to a configuration in which the hand-held terminal 5 and thebase device 6 become communicable when the hand-held terminal 5 ismounted on the base device 6. However, another embodiment may be relatedto a configuration in which the hand-held terminal 5 and the base device6 become communicable even without mounting the hand-held terminal 5 onthe base device 6. For example, when the hand-held terminal 5 and thebase device 6 can perform near field communication, a configuration maybe used in which the two become communicable in response to entering awirelessly communicable range.

The hand-held terminal 5 is mountable (or may be placeable) on the basedevice 6, and the base device 6 initiates sensing of the userinformation in response to the hand-held terminal 5 being mountedthereon (see FIG. 6). With this, since the user only has to mount thehand-held terminal 5 on the base device 6 and does not have to performan operation with respect to the base device 6 for causing the basedevice 6 to initiate sensing, the time and effort of the operation bythe user can be omitted.

The base device 6 ends the sensing in response to the user being nolonger sensed by the Doppler sensor 24 (step S20). In other words, evenwhen the hand-held terminal 5 is removed from the base device 6, thebase device 6 continues sensing of the user information (see FIG. 6).With this, since the base device 6 continues the sensing even when thehand-held terminal 5 is removed from the base device 6 for some reasonduring the sensing (e.g., when the user rolls over in bed and accidentlyhits the hand-held terminal 5), sensing of the user in sleep can beperformed with certainty. In another embodiment, the base device 6 mayend the sensing under a condition that the user has awakened. With this,similarly to the embodiment described above, the base device 6 cancontinue sensing of the user information even when the hand-heldterminal 5 is removed from the base device 6.

The information processing system 1 calculates a sleep index related tosleep of the user on the basis of the user information (biologicalinformation) (step S13), and operation and/or function of the hand-heldterminal 5 is controlled depending on the sleep index (FIG. 6, stepS15). With this, the operation and/or function of the hand-held terminal5 can be controlled depending on the sleep state of the user. Forexample, power consumption can be reduced by setting the hand-heldterminal 5 to the OFF-mode during sleep, and sleep of the user can beprevented from being disturbed by turning OFF the telephone callfunction during sleep.

With the embodiment described above, the hand-held terminal 5 iselectrically connectable to the stationary base device 6. The basedevice 6 includes charging means (the power acquisition section 23)supplies power to the hand-held terminal 5 electrically connected to thebase device 6 to charge the hand-held type device, and sensing means(the Doppler sensor 24) that senses the user information (biologicalinformation) for evaluating QOL of the user. The hand-held terminal 5transmits, to the server 3 as transmission information, information(health information, or may be user information itself) calculated fromthe user information sensed by the base device 6. The server 3accumulates, in a predetermined storage section as accumulationinformation, the transmission information (or may be informationcalculated from the transmission information) transmitted from thehand-held terminal 5.

“Electrically connectable” as described above is a meaning that includesthe following connection modes.

A mode in which, as in the embodiment described above, the hand-heldterminal is detachably/reattachably mounted (or may be placed) on thebase device, and become communicable when respective connectors makecontact.

A mode in which the hand-held terminal and the base device becomecommunicable when being connected via a detachable/reattachable cable.

A mode in which near field communication between the hand-held terminaland the base device becomes possible when the hand-held terminal isdetachably/reattachably mounted or placed on the base device.

With the modes described above, the user information can be sensedwithout requiring much installation space, since the base device 6doubles as a battery charger for the hand-held terminal 5 and a sensordevice for sensing the user information. Since the transmissioninformation is accumulated in the server 3, a large number of sets ofthe transmission information, which is useful information, can be easilymanaged without placing a large burden on the storage capacity of thehand-held terminal 5.

Since the base device 6 has the charging function, the user can bemotivated to mount the hand-held terminal 5 on the base device 6. Thus,in cases where the base device 6 initiates sensing of the userinformation in response to mounting or placing of the hand-held terminal5 (see FIG. 6), the possibility of the user forgetting to mount thehand-held terminal 5 on the base device 6 can be reduced, wherebysensing by the base device 6 can be performed with more certainty.

The charging means (the power acquisition section 23) charges thehand-held terminal 5 which is a transmission destination of the userinformation while the user information (biological information) is beingsensed. With this, the user information is sensed while the userconnects the hand-held terminal 5 to the base device 6 for charging thehand-held terminal 5. While the hand-held terminal 5 is being charged,the user is ordinarily thought to often be near the hand-held terminal5. Thus, by performing sensing of the user information when the user is(highly likely to be) near the hand-held terminal 5, the burden on theuser during the sensing can be reduced.

(Functions and Effects Related to Evaluation Based on Multiple Sets ofInformation)

In the embodiment described above, the information processing system 1includes the stationary base device 6 and the hand-held terminal 5. Thebase device 6 senses first user information (biological information) forevaluating QOL of the user while the user is asleep (steps S1, S12). Thehand-held terminal 5 senses second user information (positionalinformation, environment sensor information, information of sound sensedby the microphone, information of image captured by the camera, etc.)for evaluating QOL of the user while the user is awake (steps S5, S22,S51). The information processing system 1 performs an evaluationregarding QOL of the user on the basis of the first user information andthe second user information (steps S7, S35, S114). With theconfiguration described above, since the information processing system 1evaluates QOL by using both the first user information sense while theuser is asleep and the second user information sensed while the user isawake, the information processing system 1 can perform the evaluation ofQOL by taking into consideration the state of the user through the wholeday.

In the configuration described above, the first user information and thesecond user information may be the same or different. For example, thefirst user information and the second user information may be biologicalinformation regarding the user. Alternatively, when both types of theuser information are biological information, the first user informationand the second user information may be the same type of biologicalinformation or may be different types of biological information.

The information processing system 1 successively transmits the firstuser information (biological information, or may be informationcalculated from the first user information) to the hand-held terminal 5(step S1). The hand-held terminal 5 suspends sensing of the second userinformation while receiving information from the base device 6 (see FIG.1). With this, the hand-held terminal 5 can judge whether or not toperform sensing of the second user information by receiving informationfrom the base device 6. The hand-held terminal 5 can efficiently performthe sensing since sensing of the second user information can besuspended in a period in which the necessity thereof is low (in a periodin which the user is estimated not to be awake).

In addition, the information processing system 1 calculates a firstindex (first fatigue level/health information) that concerns QOL of theuser and is calculated on the basis of the first user information (stepS17), and calculate a second index (second fatigue level/activityinformation, environmental information, emotion information) thatconcerns QOL of the user and is calculated on the basis of the seconduser information (step S114). The information processing system 1presents the user with the first index (the sleep-state score image62/the health score image 52) and the second index (the awake-statescore image 63/the environmental score image 53 and the emotion scoreimage 54) in association with one another (FIG. 26). With this, thefirst index based on information sensed while the user is asleep and thesecond index based on information sensed while the user is awake can bepresented to the user in an easily understandable manner since the twoare associated with one another.

Furthermore, the information processing system 1 presents the user withan evaluation result (the health score image 61 or the QOL score image50) that concerns QOL and is based on the first index and the secondindex, and at least one (both in the examples shown in FIGS. 25 and 26)of the first index (the sleep-state score image 62 or the health scoreimage 52) and the second index (the awake-state score image 63 orrespective images 53 and 54) in association with one another (FIGS. 25and 26). With this, the evaluation result based on the two indices andat least one of the indices can be presented to the user in an easilyunderstandable manner since the two are associated with one another.

The hand-held terminal 5 is mountable (or placeable) on the base device6, and suspends sensing of the second user information while beingmounted (or placed) on the base device 6. With this, when the user isnot carrying the hand-held terminal 5, the hand-held terminal 5 suspendssensing of the second user information. Ordinarily, the user is thoughtto be not carrying the hand-held terminal 5 while being asleep. Thus,with the configuration described above, the possibility of the hand-heldterminal 5 sensing the second user information unnecessary while theuser is asleep can be reduced.

In the embodiment described above, the information processing system 1includes a first terminal (the base device 6), a second terminal (thehand-held terminal 5), and the server 3. The base device 6 senses thefirst user information (biological information) for evaluating QOL ofthe user (steps S1, S12). The hand-held terminal 5 senses the seconduser information (positional information, environment sensorinformation, information of sound sensed by the microphone, informationof image captured by the camera, etc.) for evaluating QOL of the user(steps S5, S22, S51). In addition, the hand-held terminal 5 transmitsinformation (health information or may be the first user informationitself) calculated from the first user information to the server 3 asthe first transmission information (steps S4, S19). Furthermore, thehand-held terminal 5 transmits, together with or separately from thefirst transmission information, information (activity information or maybe the second user information itself) calculated from the second userinformation to the server 3 as the second transmission information (stepS5, step S23, etc., step S54). One of the base device 6 and thehand-held terminal 5 performs sensing while the user is asleep, and theother performs sensing while the user is awake (see FIG. 4). The server3 performs an evaluation regarding QOL of the user on the basis of thefirst transmission information and the second transmission information(step S7, S114, or S115).

Since the information processing system 1 evaluates QOL by using boththe first user information sense while the user is asleep and the seconduser information sensed while the user is awake, the informationprocessing system 1 can perform the evaluation of QOL by taking intoconsideration the state of the user through the whole day. In addition,since the information sensed by the first terminal (the base device 6)is transmitted to the server 3 by the second terminal (the hand-heldterminal 5), the base device 6 does not have to have means tocommunicate with the server 3, and the configuration of the base device6 can be simplified. In another embodiment, the base device 6 mayinclude a communication section that performs communication with theserver 3. In this case, the second user information (or informationcalculated from the second user information) sensed by the hand-heldterminal 5 may be transmitted to the server 3 via the base device 6.

In the embodiment described above, the stationary type device (the basedevice 6) senses the first user information (biological information)which is for evaluating QOL of the user and which is informationregarding state and/or behavior of the user (steps S1, S12). Thehand-held terminal 5 senses the second user information (positionalinformation, environment sensor information, information of sound sensedby the microphone, information of image captured by the camera, etc.)that is for evaluating QOL of the user and that is information regardingstate and/or behavior of the user (steps S5, S22, S51). In addition, thehand-held terminal 5 transmits information (health information or may bethe first user information itself) calculated from the first userinformation to the server 3 as the first transmission information (stepsS4, S19). Furthermore, the hand-held terminal 5 transmits, together withor separately from the first transmission information, information(various types of QOL factor information or may be the second userinformation itself) calculated from the second user information to theserver 3 as the second transmission information (step S5, step S23,etc., step S54). The server 3 performs an evaluation regarding QOL ofthe user on the basis of the first transmission information and thesecond transmission information (steps S7, S114). It should be notedthat “information regarding state and/or behavior of the user” describedabove may be information representing the state of the user (e.g., theabove described biological information, positional information,environmental information, emotion information, and preferenceinformation), or may be information representing the behavior of theuser (e.g., the above described positional information and activityinformation).

The information processing system 1 can perform the evaluation inconsideration of the state of the user through a longer period of time,since evaluation of QOL is performed by using both the first userinformation sensed by the base device 6 and the second user informationsensed by the hand-held terminal 5. For example, it is also possiblehave the hand-held terminal 5 sense the second user information in aperiod where the user is carrying the hand-held terminal 5, and have thebase device 6 sense the first user information in a period where theuser is not carrying the hand-held terminal 5. In addition, since theinformation sensed by the base device 6 is transmitted to the server 3by the hand-held terminal 5, the base device 6 does not have to havemeans to communicate with the server 3, and the configuration of thebase device 6 can be simplified.

The base device 6 senses, as the first user information, informationregarding sleep of the user, and the hand-held terminal 5 senses, as thesecond user information, information regarding behavior content whilethe user is awake (see FIG. 4). With this, since the informationprocessing system 1 evaluates QOL by using both the first userinformation sensed while the user is asleep and the second userinformation sensed while the user is awake, the information processingsystem 1 can perform the evaluation of QOL by taking into considerationthe state of the user through the whole day.

In the embodiment described above, the information processing system 1includes a first device (the base device 6) and a second device (thehand-held terminal 5). The base device 6 senses, in a state of not beingin contact with the user, the first user information (biologicalinformation) for evaluating QOL of the user when the user is locatedindoors (steps S1, S12). The hand-held terminal 5 senses the second userinformation (positional information, environment sensor information,information of sound sensed by the microphone, information of imagecaptured by the camera, etc.) for evaluating QOL of the user when theuser is located outdoors (steps S5, S22, S51). As long as the hand-heldterminal 5 can sense the second user information when the user islocated outdoors, the hand-held terminal 5 may sense the second userinformation when the user is located indoors. The information processingsystem performs an evaluation regarding QOL of the user on the basis ofthe first user information and the second user information (steps S7,S114).

With the configuration described above, since the information processingsystem 1 evaluates QOL by using both the first user information sensedwhen user is located indoors and the second user information sensed whenthe user is located outdoors, the information processing system 1 canperform the evaluation of QOL by taking into consideration the state ofthe user through the whole day.

In the embodiment described above, the information processing system 1includes a first device (the base device 6) and a second device (thehand-held terminal 5). The base device 6 senses the first userinformation (biological information) for evaluating QOL of the userwhile the user is asleep (steps S1, S12). The hand-held terminal 5senses the second user information (positional information, environmentsensor information, information of sound sensed by the microphone,information of image captured by the camera, etc.) for evaluating QOL ofthe user while the user is awake (steps S5, S22, S51). The informationprocessing system 1 performs an evaluation regarding QOL of the user onthe basis of the first user information (calculate the first fatiguelevel at step S17, or calculate the health index at step S114), andperforms an evaluation regarding QOL of the user on the basis of thesecond user information (calculate the second fatigue level at stepS114, or calculate the environmental index or the emotion index at stepS114). Furthermore, the information processing system 1 performs anevaluation regarding QOL of the user on the basis of the first userinformation and the second user information (calculate the health indexat step S114, or calculate the QOL index at step S115).

With the configuration described above, since the information processingsystem 1 evaluates QOL by using both the first user information sensedwhile the user is asleep and the second user information sensed whilethe user is awake, the information processing system 1 can perform theevaluation of QOL by taking into consideration the state of the userthrough the whole day. The information processing system 1 can perform amultilateral evaluation regarding QOL since evaluations based on twosets of user information are performed, i.e., the evaluation based onthe first user information and the evaluation based on the second userinformation.

The information processing system 1 calculates an index regarding QOL ofthe user on the basis of the evaluation result (first fatigue level orhealth index) based on the first user information and the evaluationresult based on the second user information (second fatigue level, orenvironmental index or emotion index) (step S114 or S115). With this, aQOL index that takes into consideration both sets of user informationcan be calculated since the index regarding QOL is calculated on thebasis of the evaluation results based on the respective sets of the userinformation.

Furthermore, the hand-held terminal 5 senses the second user informationby a sensor (the position sensing section 13, more specifically, a GPSsensor) that is of a type different from the sensor (the Doppler sensor24) for sensing the first user information by the base device 6. Bysensing the respective sets of the user information with differentsensors, the user information can be acquired in a further multilateralmanner, and the accuracy of the evaluation regarding QOL can beimproved.

In the embodiment described above, the information processing system 1includes the terminal system 2 and the server 3. The terminal system 2senses the first user information (biological information) regardingsleep of the user while the user is asleep (steps S1, S12), and sensesthe second user information (QOL factor information) regarding behaviorcontent of the user while the user is awake (step S5). The server 3transmits, to the terminal system 2, service data for providing the userwith a service using the first user information and the second userinformation (step S9). With the configuration described above, a usefulservice can be provided by using the first user information regardingsleep of the user and the second user information regarding behaviorcontent of the user.

Furthermore, the server 3 evaluates QOL of the user on the basis of thefirst user information (at step S114, calculate the health index). Theinformation processing system 1 provides a service for improving QOL onthe basis of the evaluation result of QOL of the user, and selects, asservice data for providing the service, service data having a content inaccordance with preference of the user specified on the basis of thesecond user information (preference information) (step S116). With this,the information processing system can provide a useful service since theservice for improving QOL of the user can be provided with a servicecontent in accordance with the preference of the user.

(Functions and Effects Related to Privilege Bestowal in Accordance withEvaluation Result Regarding Health)

In the embodiment described above, the information processing systemincludes the server 3 and the terminal system 2 including the hand-heldtype terminal (the hand-held terminal 5). The terminal system 2 acquiresthe first information (biological information or positional information)for evaluating health of the user (step S12 or S51). The informationprocessing system performs an evaluation regarding health of the user onthe basis of the acquired first information (step S114). The server 3transmits, to the terminal system, service data for providing a networkservice in accordance with the evaluation regarding health of the user(steps 9, S36, S117). In addition, the server 3 stores, in apredetermined storage section, privilege information (privilege datashown in FIG. 21) indicating a privilege that is to be given to the userand that relates to the network service and/or the hand-held typeterminal. The server 3 is updates the privilege information such that aprivilege in accordance with a result of the evaluation regarding healthof the user is to be given to the user (step S116).

In the configuration described above, the information processing system1 gives the user a privilege in accordance with an evaluation result ofhealth of the user. With this, the information processing system 1 canprovide the user with a motivation to improve health by giving aprivilege to the user, and can provide a useful service that cancontribute in health improvement of the user.

The information processing system 1 repeatedly acquires the firstinformation regardless of whether or not an instruction by the userexists (step S12 or S51). In addition, the information processing system1 repeatedly performs an evaluation on the basis of the firstinformation (in the embodiment described above, the evaluation isrepeated every day). The server 3 continuously transmits the servicedata to the terminal system 2 regardless of whether or not aninstruction by the user exists (step S117). In addition, the server 3gives a privilege as part of a service in the network service that isprovided continuously (in the embodiment described above, transmissionof the service data is executed every day). With this, the informationprocessing system 1 automatically and continuously performs anevaluation regarding health, and continuously provides the user with anetwork service in accordance with a result of the evaluation. Thus, theuser can continuously receive an evaluation of health without repeatedlygiving instructions. By continuously performing an evaluation and givingthe user a privilege in accordance with the evaluation, the informationprocessing system 1 can contribute in health maintenance of the usersince a motivation to maintain health can be provided to the user.

In the embodiment described above, the information processing system 1includes the terminal system 2 and the server 3. The terminal system 2acquires, as the first information, biological information from the userin sleep by a sensor (the Doppler sensor 24) senses the biologicalinformation (step S12). The information processing system 1 performs anevaluation regarding health of the user on the basis of the acquiredbiological information (step S114). The server 3 stores, in apredetermined storage section, privilege information (privilege datashown in FIG. 21) indicating a privilege to be given to the user. Theserver 3 updates the privilege information such that a privilege inaccordance with a result of the evaluation regarding health of the useris to be given to the user. In the configuration described above, sincethe information processing system 1 gives the user a privilege inaccordance with an evaluation result of health of the user, theinformation processing system 1 can provide the user with a motivationto improve health, and can contribute in health improvement of the user.

The terminal system 2 further acquires second information (may bepositional information or activity information) regarding behavior ofthe user (step S91), and the information processing system 1 assessespreference of the user on the basis of the second information (stepS93). The server 3 determines, on the basis of the preference of theuser, a content of the privilege to be given to the user (step S116).With this, the information processing system can provide a usefulservice since the privilege with a content in accordance with thepreference of the user can be provided. Since the privilege can be mademore attractive to the user, a stronger motivation to improve health canbe provided to the user.

In the configuration described above, the server 3 performs anevaluation regarding health on the basis of the first information(biological information) and the second information (positionalinformation or activity information) (step S114). As a result, since theevaluation regarding health can be performed on the basis of twodifferent types of information, the accuracy of the evaluation can beimproved.

(Functions and Effects Related to Base Device)

In the embodiment described above, the terminal system 2 (the basedevice 6) which is one example of a display system has the followingconfiguration.

A sensor (the Doppler sensor 24) that senses user information(biological information) for calculating a state regarding sleep of theuser.

The projector 25 that projects and displays a predetermined image.

The control section 22 that causes the projector 25 to project anddisplay an image related to the state of sleep calculated on the basisof the user information (step S18).

The “image related to the state of sleep” may be an image representing asleep state (e.g., the sleep index described above) or may be an imagerepresenting an evaluation result of the sleep state (e.g., the pleasantsleep index number described above). With the configuration describedabove, the terminal system 2 can, by using the projector 25, project anddisplay an image at a spot easily viewable by the user who is in bed,such as, for example, the ceiling. With this, the terminal system 2 canprovide a display that is easy to view by the user in bed. In addition,the terminal system 2 can, by causing the projector 25 to display theimage, present the sleep state to the user.

In the embodiment described above, the terminal system 2 (the basedevice 6) which is one example of the display system has the followingconfiguration.

A sensor (the Doppler sensor 24) that senses user information(biological information) for calculating a state regarding sleep of theuser.

The projector 25 that projects and displays a predetermined image.

The control section 22 that specifies a timing regarding awakening ofthe user on the basis of the user information, and cause the projectorto initiate projecting and displaying an image in accordance with thespecified timing (step S14, FIG. 6).

The “timing regarding awakening of the user” is, for example, a timingwhen the user awakens, or a timing before or after the user awakens. Thetiming when a judgment is made that sleep of the user has become shallow(in other words, the user entering a state prior to awakening) and thetiming when a judgment is made that the user have awakened in theembodiment described above can be considered as the “timing regardingawakening of the user”. In addition, a timing after the user hasawakened and also based on the awakening timing (e.g., a timing reachwhen a predetermined time period has elapsed from the awakening timing)can be considered as the “timing regarding awakening of the user”.

Furthermore, “initiate (projecting and displaying an image) inaccordance with the specified timing” includes both a meaning “initiateprojecting and displaying at the timing” and a meaning “initiateprojecting and displaying at a timing depending on a specified timing(e.g., a timing one minute after the specified timing)”.

With the configuration described above, the terminal system 2 can, byusing the projector 25, provide a display that is easy to view by theuser in bed. In addition, the terminal system 2 can present the userwith an image in response to awakening of the user. Thus, the terminalsystem 2 can present an image to the user at an appropriate timing, andcan reduce power consumption by the projector 25.

In the embodiment described above, the terminal system 2 which is oneexample of the display system has the following configuration.

A sensor (the Doppler sensor 24) that senses user information forcalculating a state regarding sleep of the user in bed.

The projector 25 that projects and displays a predetermined image forthe user in bed.

The control section 22 that controls the projector 25.

The communication section 10 that performs communication with the server3 via a network.

With the configuration described above, the terminal system 2 can, byusing the projector 25, project and display an image at a spot easilyviewable by the user who is in bed, such as, for example, the ceiling.With this, the terminal system 2 can provide a display that is easy toview by the user in bed. Furthermore, since the terminal system 2includes the communication section that performs communication with theserver 3, the terminal system 2 can, for example, acquire an image to bedisplayed by the projector 25 from the server 3, or transmit, to theserver 3, a sensing result (or information based on the sensing result)from the sensor.

The control section 22 causes the projector 25 to display an imagerelated to the state of sleep calculated on the basis of the userinformation at a timing in accordance with the sleep state of the user,which is a timing specified on the basis of the user information. Withthis, an image can be presented to user at an appropriate timingdepending on the sleep state of the user. In addition, power consumptionby the projector 25 can be reduced.

In addition, the control section 22 causes the projector 25 to displayan image related to the sleep state at a timing when the user awakens ora timing before the user awakens, which is a timing specified on thebasis of the user information. With this, since information related tothe sleep state of the user can be presented to the user when the userawakens, useful information can be presented to the user at anappropriate timing.

The terminal system 2 controls power supply to the projector 25 inaccordance with the state regarding sleep. For example, the terminalsystem 2 may start power supply to the projector 25 in accordance with atiming specified on the basis of the user information regardingawakening of the user. More specifically, the terminal system 2 maystart power supply to the projector 25 at a timing before the userawakens (or a timing when the user awakens). With this, image display bythe projector 25 can be performed when the user awakens whilesuppressing power supply to the projector 25 when the user is asleep.

The terminal system 2 may change an image (content for sleep onset) forinducing sleep onset of the user and/or an image (content for awakening)for inducing awakening of the user, in accordance with the sleep stateof the user, and may cause the projector 25 to project and display theimage. For example, brightness of the image projected and displayed bythe projector 25 may be changed in accordance with the sleep state ofthe user.

When the user awakens, the terminal system 2 judges whether or not theawakening is an awakening in mid-course of sleep (see “(InformationPresentation at time of Mid-Sleep Awakening)” and “(Presentation ofEvaluation Result at time of Awakening)” described above). At this time,the terminal system 2 causes the projector 25 to project and displaydifferent images in a case where the awakening of the user is judged tobe a mid-sleep awakening and in a case where the awakening of the useris judged not to be a mid-sleep awakening (FIG. 6), respectively. Withthis, a proper image, depending on whether or not awakening of the useris a mid-sleep awakening, can be presented to the user.

The terminal system 2 performs an evaluation of sleep of the user on thebasis of the user information (step S17), and causes the projector 25 toproject and display a result of the evaluation at a timing in accordancewith the sleep state of the user (e.g., a timing when the user awakens),which is a timing specified on the basis of the user information (stepS18). With this, the terminal system 2 can present the evaluation resultof the sleep to the user at an appropriate timing.

In the embodiment described above, the terminal system 2 (the basedevice 6) which is one example of the display system has the followingconfiguration.

A sensor (the Doppler sensor 24) that senses user information(biological information) for calculating a state regarding sleep of theuser.

The projector 25 that projects and displays a predetermined image.

The control section 22 (step S14) that controls the projector inaccordance with the state of sleep that is calculated on the basis ofthe user information.

With the configuration described above, the terminal system 2 can, byusing the projector 25, project and display an image at a spot easilyviewable by the user who is in bed, such as, for example, the ceiling.With this, the terminal system 2 can provide a display that is easy toview by the user in bed. In addition, by controlling the projector 25 inaccordance with the sleep state of the user, the terminal system 2 canappropriately control the power supply (ON/OFF) of the projector 25and/or the image projected and displayed by the projector 25, inaccordance with the sleep state.

In the embodiment described above, the terminal system 2 which is oneexample of the display system has the following configuration.

A sensor (the Doppler sensor 24) that senses user information forcalculating a state regarding sleep of the user in bed.

The projector 25 that projects and displays a predetermined image forthe user in bed.

A display device (the display 17 of the hand-held terminal 5)detachably/reattachably connected to the projector 25, and having ascreen that displays an image identical to or different from thepredetermined image.

With the configuration described above, a display that is easy to viewby the user in bed can be provided by the projector 25, and the imagecan be presented by the display device at a position different from theprojection spot of the projector 25. In another embodiment, the displaydevice may be formed integrally with the projector 25.

The display device (the hand-held terminal 5) is detachable/reattachablewith respect to a casing of the projector 25. Thus, the user can use thedisplay device removed from the projector 25. Further, since the displaydevice is a hand-held type information processing apparatus, the usercan carry and use the display device. In addition, the display device iscommunicable with an external device (the server 3) via a network byaccessing the network through wireless communication. Thus, the user canuse the display device outdoors (e.g., the user can cause the displaydevice to display a content acquired from the server 3 throughcommunication).

In the embodiment described above, the base device 6 which is oneexample of the display device includes the sensor (the Doppler sensor24) for sensing the biological information of the user, and theprojector 25 that is disposed in the same casing as that for the sensor,and projects and displays a predetermined image (FIG. 2, FIG. 3). Withthis, a display that is easy to view by the user in bed can be providedby the projector 25. In addition, the biological information of the usercan be sensed by the sensor.

The Doppler sensor 24 senses biological information from a user who isaway from the base device 6 within a predetermined range. With this, thebiological information can be sensed from a user in the vicinity of thedisplay device without making the user aware of the sensing operation bythe sensor.

In addition, the base device 6 is a stationary type device which is usedin the placed state during sensing by the Doppler sensor 24. Thus, theuser only has to place the display device in a place where sensing bythe sensor is performed, and the user need not perform a troublesomesetting operation for sensing by the sensor.

The base device 6 may further include the loudspeaker 26 disposed in thesame casing as that for the Doppler sensor 24 and the projector 25. Withthis, a sound (e.g., a sound matched with the image projected anddisplayed by the projector 25) can be presented to the user in bed.Since a separated loudspeaker need not be prepared, the configuration ofthe system can be simplified.

(Functions and Effects Related to Calculation of Emotion Information)

In the embodiment described above, the hand-held terminal 5 includes thesensing section (the microphone 15 and the camera 16) that senses thesensor information for determining the emotion of the user. Theinformation processing system 1 determines the emotion of the user onthe basis of the sensor information (step S26, step S77, step S87). Thesensing section senses the sensor information in a period during whichthe hand-held terminal 5 is in a standby state.

The “standby state” is, for example, a state in which the display 17 ofthe hand-held terminal 5 is turned off (in other words, a state in whichan operation has not been performed on the hand-held terminal 5 for apredetermined time period). In the standby state, an application may bebeing execute. In the description above, the sensing section only has tobe able to sense the sensor information in the standby state. Thesensing section need not constantly sense the sensor information in thestandby state. In addition, the sensing section may sense the sensorinformation in a state (e.g., a state in which the user operates thehand-held terminal 5) different from the standby state.

Further, in the embodiment described above, the sensor information is asound sensed by the microphone 15, and an image captured by the camera16. However, the sensor information may be biological information ofuser, for example. For example, pulse of the user may be sensed as thebiological information, and the emotion of the user (whether the user isexcited or calm) can be determined on the basis of the pulse.Alternatively, for example, the sweating state of the user may be sensedwhen the user touches the hand-held terminal 5, and the emotion of theuser can be determined on the basis of the sweating state. In a casewhere the hand-held terminal 5 includes an acceleration sensor, theinformation processing system 1 may calculate the posture and/or thewalking manner of the user on the basis of a sensing result from theacceleration sensor, and may calculate the emotion information on thebasis of the posture and/or the walking manner of the user. For example,when, regarding the posture, the user walks with his/her back beinghunched more than usual, or when, regarding the walking manner, the userwalks slower than usual (plods along), the information processing system1 may calculate the emotion information indicating that the user isdepressed.

With the configuration described above, the information processingsystem 1 can determine the emotion of the user of the hand-held terminal5 on the basis of the sensor information. Since the sensor informationis sensed when the hand-held terminal 5 is in the standby state, theinformation processing system 1 can acquire the sensor information in aperiod during which the user is not aware of sensing of the sensorinformation being performed, and can determine the emotion of the userin the period. Thus, the emotion of the user, which arises in daily lifeof the user carrying the hand-held terminal 5, can be determined.

The hand-held terminal 5 acquires user-related information (scheduleinformation, positional information, and/or activity information, etc.)which is different from the sensor information and relates to the user(step S71), and controls, on the basis of the user-related information,whether or not to perform sensing of the sensor information (measurementby the microphone 15 and/or the camera 16) (step S72). With this,sensing of the sensor information can be performed in an appropriatesituation (that is, in a situation where acquisition of the sensorinformation that enables determination of the emotion is estimated).

The hand-held terminal 5 acquires information (schedule information,positional information, and/or activity information, etc.) representingthe behavior and/or activity of the user, as the user-relatedinformation. With this, the information processing system 1 can assesswhether or not to perform sensing of the sensor information, by takinginto consideration the behavior or activity of the user.

The hand-held terminal 5 judges whether or not the current time is in aperiod during which a predetermined event (e.g., meeting) related to theuser is performed, on the basis of the user-related information. Thesensor information is sensed in the period during which thepredetermined event is performed. With this, the information processingsystem 1 can determine the emotion of the user in the period of theevent such as meeting or movement, for example.

The sensing section is operable in at least a first sensing mode (astate in which a process loop of steps S81 to S83 is executed) forsensing the sensor information at a relatively long time interval, and asecond sensing mode (a state in which a process loop of steps S85 andS86 is executed) for continuously sensing the sensor information. Thehand-held terminal 5 switches the operation mode of the sensing sectionto the second sensing mode in a case where the sensor information sensedin the first sensing mode satisfies a predetermined condition (in a casewhere a judgment result in step S82 is positive). The informationprocessing system 1 determines the emotion of the user on the basis ofthe sensor information sensed in the second mode (step S87). With theconfiguration described above, the information processing system 1 cancontrol whether or not to perform sensing of the sensor information byusing the sensor information, without using the above-describeduser-related information.

The second sensing mode may be a mode in which the sensor information issensed at a relatively short time interval (as compared to the firstsensing mode).

The hand-held terminal 5 judges whether or not the user is operating thehand-held terminal (step S72). The sensing section senses the sensorinformation when the user is judged to be operating the hand-heldterminal 5 (step S73). With this, the emotion of the user who isoperating the hand-held terminal 5 can be determined (in addition to theemotion of the user when the hand-held terminal 5 is in the standbystate).

In the embodiment described above, the information processing system 1senses the sensor information for determining the emotion of the user(step S74, step S85), determines the emotion of the user on the basis ofthe sensor information (step S77, step S87), and presents the user withsuggestion information (service data) for performing suggestion to theuser in accordance with a determination result of the emotion (stepS117, step S104). With the configuration described above, theinformation processing system 1 can determine the emotion of the user onthe basis of the sensor information to acquire useful information, thatis, the emotion of the user. In addition, the information processingsystem 1 can perform, for the user, suggestion taking into considerationthe emotion of the user.

The information processing system 1 presents the suggestion informationonce a predetermined period of time (once a day) (step S9), andgenerates the suggestion information in accordance with thedetermination result of the emotion based on the sensor informationsensed in the predetermined period of time. With this, suggestion inaccordance with the determination result of the emotion can beperiodically performed for the user.

In another embodiment, the information processing system 1 may sense thesensor information in a period (a period from step S24 to step S25)during which an event is performed, and may present the user with thesuggestion information in response to the end of the period of theevent. With this, suggestion in accordance with the determination resultof the emotion can be performed for the user after the event has ended.Thus, the suggestion can be performed at an appropriate timing.

The server 3 stores, in a predetermined storage section, identificationinformation (notification user data) of another user different from theuser for which determination of emotion has been performed, as a user tobe presented with the suggestion information, in association with theuser for which determination of emotion has been performed. The server 3presents information (emotion information) indicating a determinationresult of emotion of a certain user and suggestion information (emotionindex) in accordance with the determination result, to another user(notification-subject user) associated with the certain user (stepS117). With this, the information processing system 1 can perform, foranother user different from the user himself/herself who is the subjectof emotion determination, suggestion in accordance with the emotion ofthe user himself/herself. For example, since information related to theemotion of a user is useful information for family, friends, and thelike of the user, the useful information can be provided to the family,friends, and the like.

The server 3 presents, as suggestion information in accordance with adetermination result of emotion of a certain user, a first suggestioninformation (information shown in FIG. 25) to the certain user, andpresents a second suggestion information (information shown in FIG. 28)different from the first suggestion information to another userassociated with the certain user.

Further, in the embodiment described above, the hand-held terminal 5repeatedly senses the sensor information for determining the emotion ofthe user (step S74, step S85), and determines the emotion of the user ina predetermined monitoring period (measurement period) on the basis of aplurality of sensor information sensed during the monitoring period(step S77, step S87). The information processing system 1 presents thedetermination result of the emotion to the user. The monitoring periodmay be set in any way, and any setting method may be adopted. Themonitoring period may be, for example, a period from a predeterminedstart time to a predetermined end time.

With the configuration described above, the information processingsystem 1 can determine the emotion of the user of the hand-held terminal5 on the basis of the sensor information. In addition, since the sensorinformation is sensed during the monitoring period, the informationprocessing system 1 can determine the emotion of the user during aspecific monitoring period in the daily life of the user.

The hand-held terminal 5 acquires user-related information (scheduleinformation, positional information and/or activity information, etc.)which is different from the sensor information and relates to the user,and determines the monitoring period on the basis of the user-relatedinformation. With this, for example, a period (e.g., meeting time ormeal time) during which the user conducts a certain behavior can bedetermined as the monitoring period, whereby a period for sensing thesensor information can be appropriately set.

The hand-held terminal 5 judges whether or not the user is operating thehand-held terminal 5 (steps S72, S76). The information processing system1 performs determination of the emotion for, as the monitoring period, aperiod during which the user is judged to be operating the hand-heldterminal 5 (steps S74, S77). With this, the emotion of the user can bedetermined while the user is operating the hand-held terminal 5. Inaddition, when the hand-held terminal 5 includes a camera, the face ofthe user can be captured by the camera if the user is operating thehand-held terminal 5. Thus, the emotion of the user can be determinedfrom the facial expression of the user.

In the embodiment described above, the information processing system 1senses the sensor information for determining the emotion of the user(step S74, step S85), and determines the emotion of the user on thebasis of the sensor information (step S77, step S87). In addition, theinformation processing system 1 acquires life information (positionalinformation, activity information, health information, environmentsensor information, environmental information, and preferenceinformation) which is obtained by observing the life of the user, andexecutes a predetermined process (evaluation of QOL, and specificationof service content in a network service) on the basis of a determinationresult of emotion, and the life information (steps S115, S116).

The “life information” may be any information obtained by observing thelife of the user. The life information may be some of theabove-described positional information, activity information, healthinformation, environment sensor information, environmental information,and preference information described above, or may be other types ofinformation. In addition, the “predetermined process” may have anycontent. The predetermined process may be any other process than theprocess of performing evaluation related to the user and the process ofspecifying the service content which are described above.

With the configuration described above, the information processingsystem 1 can determine the emotion of the user of the hand-held terminal5 on the basis of the sensor information. In addition, the informationprocessing system 1 can perform a process reflecting the life andemotion of the user, by using the determination result of the emotionand the life information.

The information processing system 1 executes, as the predeterminedprocess described above, a process of calculating an index indicatingthe state of the user on the basis of the determination result of theemotion and the life information (step S115). With this, the informationprocessing system 1 can calculate a useful index reflecting the QOL ofthe user.

The information processing system 1 executes, as the above-describedpredetermined process, a process of generating suggestion information(advice information, recommendation information) for performingsuggestion to the user, on the basis of the determination result of theemotion and the life information (step S116), and presenting thesuggestion information to the user (step S117). With this, usefulsuggestion can be presented to the user, taking into consideration thelife and emotion of the user.

The information processing system 1 controls whether or not to performsensing of the sensor information, on the basis of the life information,and calculates an index (emotion index, QOL index, etc.) indicating thestate of the user. With this, the information processing system 1 canassess whether or not to perform sensing of the sensor information, onthe basis of the life information, whereby this assessment can beappropriately performed.

The information processing system 1 acquires the life information in aperiod at least a part of which overlaps with the period during whichthe sensor information is sensed (FIG. 10). With this, the informationprocessing system 1 can perform a process reflecting the emotion of theuser in a certain period, and the life information of the user in thisperiod.

In the embodiment described above, the hand-held terminal 5 includes thesensing section (the microphone 15 and the camera 16) which senses thesensor information for determining the emotion of the user. Theinformation processing system 1 determines the emotion of the user onthe basis of the sensor information, and accumulates a result of thedetermination of the emotion in the storage section included in theserver 3.

The hand-held terminal 5 identifies the voice of the user included inthe sound sensed by the microphone 15, and determines the emotion of theuser on the basis of the voice of the user (step S77, step S87). Withthis, since the emotion of the user is determined on the basis of thevoice of the user himself/herself of the hand-held terminal 5, theemotion can be accurately determined.

The hand-held terminal 5 may identify the voice of a person other thanthe user, which voice is included in the sound sensed by the microphone15, and may determine the emotion of the user on the basis of the voiceof the other person (step S77, step S87). With this, since the emotionof the user of the hand-held terminal 5 can be determined on the basisof the voice of a person other than the user, the emotion of the usercan be determined even in a situation where the emotion of the usercannot be determined from his/her voice. Thus, the emotion of the usercan be determined in more situations.

The hand-held terminal 5 may determine the emotion of the user inaccordance with the surrounding atmosphere of the user, on the basis ofthe sound sensed by the microphone 15. With this, the emotion of theuser of the hand-held terminal 5 can be determined even in a situationwhere the emotion of the user cannot be determined from his/her voice.Thus, the emotion of the user can be determined in more situations.

With the configuration described above, the information processingsystem 1 can determine the emotion of the user of the hand-held terminal5 on the basis of the sensor information. In addition, the emotioninformation is useful information since various types of services can beprovided to the user by analyzing or evaluating the emotion information,and becomes further useful information if there are multiple sets of theemotion information (which may be multiple sets of the emotioninformation for a single user, or multiple sets of the emotioninformation for multiple users). In the embodiment described above,since the calculated emotion information is accumulated in the server 3,a large number of sets of the emotion information, which is usefulinformation, can be easily managed without placing a large burden on thestorage capacity of the hand-held terminal 5.

8. Modifications

(Modifications Related to Configuration of Terminal System)

In the embodiment described above, the terminal system 2 is configuredto include one hand-held terminal 5 and one base device 6. In anotherembodiment, the terminal system 2 may be configured to include aplurality of hand-held terminals 5 and one base device 6. That is, theterminal system 2 may have a configuration in which a plurality of users(e.g., a plurality of users in a family) each carry the hand-heldterminal 5, and only one base device 6 is provided for the plurality ofhand-held terminals 5. In this case, the base device 6 may sense thebiological information of one specific user (e.g., a user of thehand-held terminal 5 connected to the base device 6). When the basedevice 6 is able to sense the biological information from the pluralityof users, the base device 6 may sense the biological information of eachuser.

In the embodiment described above, the hand-held terminal 5 can be usedin combination with any base device 6. That is, in the embodimentdescribed above, since the information sensed by the base device 6 istransmitted to the server 3 through the hand-held terminal 5, thisinformation can be specified to be information related to the user ofthe hand-held terminal 5. Thus, as long as the relationship between thehand-held terminal 5 and the user is fixed, the base device 6 can beused by any user (any hand-held terminal 5). For example, when the basedevice 6 is placed in a hotel room as well as at the home of the user,the information processing system 1 can calculate the health informationalso when the user sleeps in places other than the home.

In another embodiment, the information processing system 1 may fix thecombination of the hand-held terminal 5 and the base device 6. Forexample, by performing an authentication process when connecting thehand-held terminal 5 and the base device 6, only a specific base device6 can be made available for a specific hand-held terminal 5.

(Modification Related to Function of Each Apparatus Included inInformation Processing System 1)

In the embodiment described above, each of the processes in theinformation processing system 1 is executed by the base device 6, thehand-held terminal 5, and the server 3. Each of the processes in theinformation processing system 1 may be executed by any of the threecomponents (the base device 6, the hand-held terminal 5, and the server3).

The base device 6 transmits the information sensed by the sensor to thehand-held terminal 5. In another embodiment, the base device 6 maytransmit information calculated from the sensed information. Forexample, the base device 6 may transmit calculated biologicalinformation regarding respiration and/or body movement to the hand-heldterminal 5, or may transmit a calculated sleep index to the hand-heldterminal 5, or may transmit calculated health information (a sleep indexand a fatigue index) to the hand-held terminal 5. Further, the basedevice 6 may transmit (an index representing) a calculated evaluationresult based on the health information to the hand-held terminal 5.

The hand-held terminal 5 may transmit information received from the basedevice 6 and/or information sensed by itself to the server 3, or maytransmit information calculated from these pieces of information to theserver 3. In the latter case, in the embodiment described above, thehand-held terminal 5 calculates the QOL factor information and transmitsthe QOL factor information to the server 3. In another embodiment, thehand-held terminal 5 may perform evaluation based on the QOL factorinformation and transmit (an index representing) an evaluation result tothe server 3.

The server 3 may receive, from the hand-held terminal 5, informationsensed by the base device 6 and/or the hand-held terminal 5. In thiscase, the server 3 calculates an evaluation result on the basis of thesensed information. In addition, the server 3 may calculate anevaluation result on the basis of information (biological informationregarding respiration and/or body movement, a sleep index, and QOLfactor information, etc.) calculated from the sensed information, or mayreceive the evaluation result from the hand-held terminal 5.

In a case where multiple types of QOL factor information are calculatedor multiple types of evaluations are performed as in the embodimentdescribed above, some QOL factor information (or some evaluationresults) among the multiple types of QOL factor information (or multipleevaluation results) may be calculated by the hand-held terminal 5, whilethe remaining QOL factor information (or remaining evaluation results)may be calculated by the server 3.

(Modification Related to Base Device)

In the embodiment described above, the base device 6 is configured to beconnected to the network 4 via the hand-held terminal 5. In anotherembodiment, the base device 6 may be configured to include acommunication section communicable with the network 4. In this case, thebase device 6 may communicate with the server 3 without involving thehand-held terminal 5. For example, the base device 6 may transmitinformation sensed by the sensor (or information calculated from theinformation) to the server 3 without involving the hand-held terminal 5.In addition, for example, the base device 6 may receive service datafrom the server 3 without involving the hand-held terminal 5. Further,the base device 6 may cause the projector 25 to project and display animage (e.g., the content for sleep onset or the content for awakeningdescribed above) on the basis of the service data, and/or an imagetransmitted from the server 3.

In the embodiment described above, the base device 6 is configured toinclude the projector 25 and the loudspeaker 26 which is one example ofthe output device. In another embodiment, the base device 6 may includeanother configuration as the output device. For example, the base device6 may include a light emitting device that emits light. The lightemitting device is a light source that irradiates the user with light,and may be an illumination device, for example. The illumination devicemay be integrated with or separated from the main body of the basedevice 6.

The terminal system 2 may control the illumination device in accordancewith the sleep state of the user. For example, the illumination devicemay be lit up when the user has entered a state prior to awakening (orwhen the user has awakened). Specifically, the terminal system 2 startslighting of the illumination device (irradiation of the light source) ata timing before the user awakens, which is a timing specified on thebasis of the biological information. With this, the user can beencouraged to wake up by irradiating the user in the state prior toawakening with light. In the description above, after the illuminationdevice is lit up, the terminal system 2 may control the illuminationdevice to gradually increase the brightness. Alternatively, the terminalsystem 2 may encourage the user to wake up by light emission of theprojector 25, instead of (or as well as) encouraging the user to wake upby light emission of the illumination device.

A point in time when the user awakens may be calculated (specified) byany method other than calculation based on the biological information.For example, when user has set the time of an alarm clock with respectto the terminal system 2 (the hand-held terminal 5), the terminal system2 may use the time as a point in time when the user awakens. Thus, theterminal system 2 which is one example of the display system mayinclude: the projector that projects and displays an image related tothe sleep of the user; the light source that irradiates the user withlight; and the control section that specifies a point in time when theuser awakens and starts irradiation of the light source at a timingbased on the point in time. With this, a display that is easy to view bythe user in bed can be provided by the projector 25, and the user in thestate prior to awakening can be encouraged to wake up by beingirradiated with light.

In another embodiment, a service to be provided as a network service maybe performed (provided) in the base device 6. That is, the base device 6may perform a predetermined output on the basis of service data receivedby the hand-held terminal 5. This predetermined output may be at leastone of image, light, sound, and odor, for example. For example, theserver 3 may transmit the content for sleep onset and/or the content forawakening described above, as service data to the hand-held terminal 5,and these contents may be reproduced by the projector 25 and/or theloudspeaker 26 of the base device 6. That is, the projector 25 mayproject and display an image based on the service data. The loudspeaker26 may output a sound based on the service data.

For example, the base device 6 may include an odor generation devicethat generates odors. In this case, the odor generation device may becontrolled on the basis of the service data received by the hand-heldterminal 5. Specifically, the service data indicating the type of anodor is received by the hand-held terminal 5. Thereafter, when thehand-held terminal 5 becomes communicable with the base device 6, thebase device 6 controls the odor generation device to output the odor ofthe type indicated by the service data. Thus, the terminal system 2 canprovide the user with the odor in accordance with the evaluation resultof the day during sleep, thereby providing the user with comfortablesleep onset environment.

In another embodiment, the base device 6 may include an environmentsensor. The environment sensor may sense at least one of temperature,humidity, illumination intensity, atmospheric pressure, and sound,similarly to the environment sensor 14 included in the hand-heldterminal 5. In this case, the base device 6 senses the biologicalinformation by using the Doppler sensor 24, and senses the environmentsensor information by using the environment sensor. A period duringwhich the environment sensor information is sensed only has to includeat least a part of the sleeping period of the user, and may be the sameas the biological information sensing period or may be the sleepingperiod (period from sleep onset to awakening) of the user. In addition,the base device 6 transmits the sensed environment sensor information tothe hand-held terminal 5. The base device 6 may transmit the environmentsensor information as it is to the hand-held terminal 5, or may processthe environment sensor information in some way and transmit theprocessed result to the hand-held terminal 5.

The hand-held terminal 5 transmits, to the server 3, the environmentsensor information from the base device 6 and/or information calculatedfrom the environment sensor information. For example, the hand-heldterminal 5 may calculate environmental information (e.g., theenvironmental value described above) from the environment sensorinformation, and transmit the environmental information to the server 3.The hand-held terminal 5 may transmit the environmental informationtogether with the health information (the environmental informationbeing included in the transmission information including the healthinformation) to the server 3, or may transmit the environmentalinformation separately from (at a timing different from that for) thehealth information to the server 3.

When the base device 6 includes the environment sensor, an output fromthe base device 6 (or the hand-held terminal 5) may be controlled on thebasis of the sensing result (environment sensor information) obtained bythe environment sensor. This control may be performed by the controlsection 22 of the base device 6, or may be performed by the processingsection 11 of the hand-held terminal 5. For example, when theenvironment sensor includes an illumination sensor, the brightness of animage projected and displayed by the projector 25 (or the display 17 ofthe hand-held terminal 5) may be controlled on the basis of the sensingresult obtained by the illumination sensor. That is, when the brightness(luminance) of a room which is sensed by the illumination sensor ishigh, the brightness of the image projected and displayed by theprojector 25 may be increased. In addition, for example, when theenvironment sensor includes a noise sensor, the output sound volume ofthe loudspeaker 26 (or the loudspeaker 18 of the hand-held terminal 5)may be controlled on the basis of the sensing result of the noisesensor. That is, when the noise sensed by the noise sensor is large, theoutput sound volume of the loudspeaker 26 may be increased.

In another embodiment, the terminal system 2 which is one example of anoutput system may include a sensor (the Doppler sensor 24) that sensesuser information (biological information) for calculating a stateregarding sleep of the user in bed, and an indirectional loudspeakerthat outputs a sound for the user in bed. The indirectional loudspeakermakes the user less likely to be conscious of the position of the sourceof the output sound, thereby providing the user with more natural sound.The indirectional loudspeaker may be control to perform similar soundoutput to that of the loudspeaker 26 described above. For example, theterminal system 2 may control the indirectional loudspeaker inaccordance with the state regarding sleep which is calculated on thebasis of the user information (biological information). Theindirectional loudspeaker may be housed in the same casing as that forthe sensor, or may be separated from the sensor.

As described above, the terminal system 2 which is one example of thedisplay system may include: the projector that projects and displays animage (the content for sleep onset, the content for awakening, the imageshowing the evaluation result of sleep, etc.) related to the sleep ofthe user; and the indirectional loudspeaker that outputs a sound (thecontent for sleep onset, the content for awakening, the voice showingthe evaluation result of sleep, etc.) related to the sleep of the user.With this, a display that is easy to view by the user in bed can beprovided by the projector 25, and more natural sound can be provided tothe user by the indirectional loudspeaker.

(Modification Related to Sensor)

In the embodiments and modifications described above, the sensor thatacquires the biological information of the user is a non-contact type(and unworn type) sensor (specifically, a Doppler sensor). In otherwords, the sensor is placed in the vicinity of a user who is a subjectto be sensed. Thus, the user need not worry about the sensing operationby the sensor, whereby the burden on the user, which is caused bysensing, can be reduced. In addition, the unworn type sensor avoids asituation that the user forgets to wear the sensor and sensing is notperformed.

The non-contact type (and unworn type) sensor is not limited to theDoppler sensor 24 described above. That is, in the embodiment describedabove, the Doppler sensor 24 emits radio waves toward the subject to besensed and receives reflected waves, and outputs the biologicalinformation on the basis of received results (by analyzing the movementof the subject on the basis of the Doppler phenomenon of the receivedwaves). In another embodiment, for example, the non-contact type sensormay emit radio waves toward the subject to be sensed and receivereflected waves, and may output the biological information on the basisof received results (received waves). Specifically, the sensor mayacquire an image of the subject on the basis of the received waves,analyze the movement of the subject from the image information, andoutput the biological information. The non-contact type sensor may be asensor that uses predetermined sensing waves such as ultrasonic wavesinstead of radio waves. The non-contact type sensor may be imaging means(camera) placed in the vicinity of the user. That is, the camera maycapture an image of the subject from a distant location, and theinformation processing system may analyze the movement of the subjectfrom the captured image to output the biological information.

(Modification Related to Network Service)

In another embodiment, a game may be provided in the server 3. A serverproviding a game may be the same server as the server 3, or may be adedicated server (game server). For example, a game application may beprovided to the user in the above-described website (website forproviding a network service) managed by the server 3. The gameapplication may be of a type that operates on a browser for browsing awebsite, or may be of a type that is downloaded from the server andinstalled in the hand-held terminal 5.

The game application may be provided from the server to the terminalsystem 2, or may be acquired by the terminal system 2 with any method.For example, a storage medium that stores the game application may beconnected (or mounted) to the hand-held terminal 5 to execute the gameapplication in the hand-held terminal 5.

The game application may be executed in the hand-held terminal 5, or maybe executed in other user terminals (a hand-held terminal, a personalcomputer, and a game apparatus, etc.) possessed by the user.

As described above, when the game application is executed in the userterminal (the hand-held terminal 5 and/or the other user terminalsdescribed above), an evaluation result of the user may be reflected inthe game. That is, as a privilege to be given to the user in theprocesses at steps S116 and S117, a privilege related to the game may begiven to the user. For example, the server 3 may give the privilegerelated to the game to the user under a condition that a state in whicha value indicated by a predetermined index is a good value equal to orlarger than a reference value has continued for a predetermined period,or a condition that a predetermined index has been improved from apredetermined standard. The privilege related to the game may be anyfavor. For example, the server 3 may give an item to be utilized in thegame to the user, or may advance a story in the game.

In the description above, the information processing system includingthe terminal system and the server executes a game process based on agame application. Here, the information processing system refers to theevaluation result (which may be information for calculating theevaluation result, that is, the user status data shown in FIG. 21)stored in the server 3 when executing the game process. Then, on thebasis of the referred evaluation result (that is, in accordance with thereferred result), a predetermined process in the game process isexecuted. The predetermined process is, for example, a process of givingthe item described above, or a process of advancing the story in thegame. Specifically, the information processing system performs a processof adding or updating the game data to be used in the game application.That is, a flag in the game is updated so that game data of the givenitem can be added, or a new story can be played.

The game process may be executed in the server-side informationprocessing apparatus (e.g., the game server or the server 3), or may beexecuted in the terminal-side information processing apparatus (e.g.,the hand-held terminal 5 or the other user terminals described above).Alternatively, the game process may be executed by the server-sideinformation processing apparatus and the terminal-side informationprocessing apparatus in cooperation with each other. That is, one partof the game process may be executed on the server side while the otherpart of the game process may be executed on the terminal side. Forexample, when the game application is operated on a browser for browsinga website, the game process is executed on the server side, or executedon the server side and the terminal side in cooperation with each other.When the game application is installed in the terminal-side informationprocessing apparatus, or when the game application stored in the storagemedium connected to the information processing apparatus is executed,the game process is executed on the terminal side, or executed on theserver side and the terminal side in cooperation with each other.

Addition or update of game data may be executed on the server side, ormay be executed on the terminal side. That is, a process of adding orupdating game data may be executed by the server-side informationprocessing apparatus performing addition or update for the game datastored in the server-side or terminal-side storage section.Alternatively, the process of adding or updating game data may beexecuted by the terminal-side information processing apparatusperforming addition or update for the game data stored in theterminal-side storage section.

When the evaluation result is used (referred) in the game application asdescribed above, the evaluation result related to the user of the gameapplication is referred to. For example, when the game application isused in the state where the user logs in a website managed by the server3, the server can specify the user by using user identificationinformation (user identification data shown in FIG. 21) that is inputtedat the time of log-in. That is, the server stores therein the useridentification information inputted at the time of log-in, and refers tothe evaluation result of the user specified by the stored useridentification information when the server refers to the evaluationresult in the game process.

When the game application is used in the state where the user does notlog in the portal website (or when the game application is used by usinga terminal different from a terminal that logs in), the serveridentifies the user who uses the game application before the gameprocess or during the game process. Specifically, the user terminal thatuses the game application receives, from the user, an input of the useridentification information before or during the game process, andtransmits the inputted user identification information to the server.The server specifies an evaluation result to be referred to, by usingthe user identification information from the user terminal. Whenaddition or update of game data is executed on the server side, theserver refers to the specified evaluation result, and executes apredetermined process (addition or change of game data) in the gameprocess on the basis of the referred evaluation result. On the otherhand, when addition or update of game data is executed on the terminalside, the server transmits the specified evaluation result to the userterminal. The user terminal refers to the received evaluation result(that is, refers to the evaluation result stored on the server side),and executes a predetermined process in the game process on the basis ofthe referred evaluation result.

In the description above, the user identification information may be notonly the account of the user in the network service according to theembodiment described above, but also an account in the service thatprovides the game application. In this case, sharing the account of theuser among a plurality of network services allows the user to use theplurality of network services in different user terminals (which may beterminals having different platforms), thereby improving usability.

As described above, an objective of the embodiment described above is,for example, to provide a user with useful information and/or service,and the embodiment can be applied to, for example, an informationprocessing system including a hand-held terminal.

While certain example systems, methods, devices and apparatuses havebeen described herein, it is to be understood that the appended claimsare not to be limited to the systems, methods, devices and apparatusesdisclosed, but on the contrary, are intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims.

What is claimed is:
 1. A display system, comprising: a sensor whichsenses user information for calculating a sleep state of a user; aprojector which projects and displays a predetermined image; and controlmeans for causing the projector to project and display an image relatedto the sleep state calculated on the basis of the user information.
 2. Adisplay system, comprising: a sensor which senses user information forcalculating a sleep state of a user; a projector which projects anddisplays a predetermined image; timing specifying means for specifying atiming regarding awakening of the user on the basis of the userinformation; and control means for causing the projector to startprojection and display of the image in accordance with the specifiedtiming.
 3. A display system, comprising: a sensor which senses userinformation for calculating a sleep state of a user in bed; a projectorwhich projects and displays a predetermined image when the user is inbed; control means for controlling the projector; and communicationmeans for performing communication with a server through a network. 4.The display system according to claim 1, wherein the control meanscauses the projector to display the image related to the sleep statecalculated on the basis of the user information, at a timing inaccordance with the sleep state of the user, which is a timing specifiedon the basis of the user information.
 5. The display system according toclaim 4, wherein the control means causes the projector to display theimage related to the sleep state at a timing before the user awakens ora timing when the user awakens, which is a timing specified on the basisof the user information.
 6. The display system according to claim 1,wherein the control means controls power supply to the projector inaccordance with the sleep state of the user.
 7. The display systemaccording to claim 6, wherein the control means starts power supply tothe projector in accordance with a timing regarding awakening of theuser, which is a timing specified on the basis of the user information.8. The display system according to claim 1, wherein the control meanschanges an image for inducing sleep onset of the user and/or an imagefor inducing awakening of the user, in accordance with the sleep stateof the user, and causes the projector to project and display the image.9. The display system according to claim 1, further comprising judgmentmeans for judging, when the user awakens, whether or not the awakeningis an awakening in mid-course of sleep, wherein the control means causesthe projector to project and display different images in a case wherethe awakening of the user is judged to be an awakening in mid-course ofsleep and in a case where the awakening of the user is judged not to bean awakening in mid-course of sleep, respectively.
 10. The displaysystem according to claim 1, further comprising evaluation means forperforming an evaluation of sleep of the user on the basis of the userinformation, wherein the control means causes the projector to projectand display a result of the evaluation at a timing in accordance withthe sleep state of the user, which is a timing specified on the basis ofthe user information.
 11. The display system according to claim 1,further comprising a light source which irradiates the user with light,wherein the control means starts irradiation of the light source at atiming before the user awakens, which is a timing specified on the basisof the user information.
 12. The display system according to claim 1,wherein the control means successively acquires the user informationfrom the sensor, and controls the projector in real time in accordancewith the acquired user information.
 13. The display system according toclaim 3, wherein the communication means receives, from the server,service data for providing the user with a network service in accordancewith an evaluation regarding health of the user.
 14. The display systemaccording to claim 13, wherein the projector projects and displays animage based on the service data.
 15. The display system according toclaim 3, wherein the communication means transmits, to the server, theuser information sensed by the sensor and/or information calculated fromthe user information.
 16. The display system according to claim 3,wherein the communication means receives an image content from theserver, and the projector projects and displays the image content. 17.The display system according to claim 1, wherein the projector projectsand displays the image on a ceiling above the projector itself.
 18. Thedisplay system according to claim 1, wherein the sensor sensesbiological information of the user in a state of not being in contactwith the user.
 19. The display system according to claim 18, wherein thesensor senses the biological information from the user who is away fromthe display device within a predetermined range.
 20. The display systemaccording to claim 19, wherein the sensor emits radio waves or soundwaves toward a subject to be sensed and receives reflected waves, andoutputs the biological information on the basis of a result of thereception.
 21. The display system according to claim 1, furthercomprising: a camera which captures a projection spot of the projector;and correction means for performing, on the image projected anddisplayed by the projector, a correction that takes into considerationunevenness of the projection spot, on the basis of an image of theprojection spot captured by the camera.
 22. The display system accordingto claim 21, wherein the correction means performs, on the imageprojected and displayed by the projector, a correction that takes intoconsideration color of the projection spot, on the basis of the image ofthe projection spot.
 23. The display system according to claim 1,further comprising a loudspeaker provided in the same casing as that forthe sensor and the projector.
 24. A display device, comprising: a sensorwhich senses user information for calculating a sleep state of a user; aprojector which projects and displays a predetermined image; and controlmeans for causing the projector to project and display an image relatedto the sleep state calculated on the basis of the user information. 25.A display device, comprising: a sensor which senses user information forcalculating a sleep state of a user; a projector which projects anddisplays a predetermined image; timing specifying means for specifying atiming regarding awakening of the user on the basis of the userinformation; and control means for causing the projector to startprojection and display of the image in accordance with the specifiedtiming.
 26. A display device, comprising: a sensor which senses userinformation for calculating a sleep state of a user in bed; a projectorwhich projects and displays a predetermined image when the user is inbed; control means for controlling the projector; and communicationmeans for performing communication with a server through a network.